{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 题目：分类器练习"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "任务描述 ：\n",
    "请在 Pima Indians Diabetes Data Set（皮马印第安人糖尿病数据集）进行分类器练习。 \n",
    " \n",
    "需要提交代码文件，并给出必要的结果解释。 \n",
    "1)训练数据和测试数据分割（随机选择 20%的数据作为测试集）；（10 分） \n",
    "2)适当的特征工程（及数据探索）;（10 分） \n",
    "3)Logistic 回归，并选择最佳的正则函数（L1/L2）及正则参数；（30 分） \n",
    "4)线性 SVM，并选择最佳正则参数，比较与 Logistic 回归的性能，简单说明原因。\n",
    "（20 分） \n",
    "5)RBF 核的 SVM，并选择最佳的超参数（正则参数、RBF 核函数宽度）；（30 分） \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1 导入必要的包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 首先 import 必要的模块\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "#竞赛的评价指标为logloss\n",
    "from sklearn import metrics\n",
    "from sklearn.metrics import log_loss\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 读取数据&的数据探索"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pregnancies</th>\n",
       "      <th>Glucose</th>\n",
       "      <th>BloodPressure</th>\n",
       "      <th>SkinThickness</th>\n",
       "      <th>Insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>DiabetesPedigreeFunction</th>\n",
       "      <th>Age</th>\n",
       "      <th>Outcome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6</td>\n",
       "      <td>148</td>\n",
       "      <td>72</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>33.6</td>\n",
       "      <td>0.627</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>85</td>\n",
       "      <td>66</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>26.6</td>\n",
       "      <td>0.351</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "      <td>183</td>\n",
       "      <td>64</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23.3</td>\n",
       "      <td>0.672</td>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>89</td>\n",
       "      <td>66</td>\n",
       "      <td>23</td>\n",
       "      <td>94</td>\n",
       "      <td>28.1</td>\n",
       "      <td>0.167</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>137</td>\n",
       "      <td>40</td>\n",
       "      <td>35</td>\n",
       "      <td>168</td>\n",
       "      <td>43.1</td>\n",
       "      <td>2.288</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pregnancies  Glucose  BloodPressure  SkinThickness  Insulin   BMI  \\\n",
       "0            6      148             72             35        0  33.6   \n",
       "1            1       85             66             29        0  26.6   \n",
       "2            8      183             64              0        0  23.3   \n",
       "3            1       89             66             23       94  28.1   \n",
       "4            0      137             40             35      168  43.1   \n",
       "\n",
       "   DiabetesPedigreeFunction  Age  Outcome  \n",
       "0                     0.627   50        1  \n",
       "1                     0.351   31        0  \n",
       "2                     0.672   32        1  \n",
       "3                     0.167   21        0  \n",
       "4                     2.288   33        1  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 读取数据\n",
    "# path to where the data lies\n",
    "dpath = './data/'\n",
    "train = pd.read_csv(\"diabetes.csv\")\n",
    "train.head()\n",
    "#----------------------------\n",
    "#输入变量\n",
    "#Pregnancies： 怀孕次数 \n",
    "#Glucose： 口服葡萄糖耐受试验中，2 小时的血浆葡萄糖浓度。 \n",
    "#BloodPressure： 舒张压（mm Hg） \n",
    "#SkinThickness： 三头肌皮肤褶层厚度（mm） \n",
    "#Insulin：2 小时血清胰岛素含量（μU/ ml） \n",
    "#BMI： 体重指数（体重，kg /（身高，m）^ 2） 2) \n",
    "#DiabetesPedigreeFunction： 糖尿病家族史 3) \n",
    "#Age： 年龄（岁） \n",
    "#-----------------------------\n",
    "#输出变量\n",
    "#Outcome： 输出变了/类别标签（0 或 1，出现糖尿病为 1, 否则为 0） "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pregnancies</th>\n",
       "      <th>Glucose</th>\n",
       "      <th>BloodPressure</th>\n",
       "      <th>SkinThickness</th>\n",
       "      <th>Insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>DiabetesPedigreeFunction</th>\n",
       "      <th>Age</th>\n",
       "      <th>Outcome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>763</th>\n",
       "      <td>10</td>\n",
       "      <td>101</td>\n",
       "      <td>76</td>\n",
       "      <td>48</td>\n",
       "      <td>180</td>\n",
       "      <td>32.9</td>\n",
       "      <td>0.171</td>\n",
       "      <td>63</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>764</th>\n",
       "      <td>2</td>\n",
       "      <td>122</td>\n",
       "      <td>70</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>36.8</td>\n",
       "      <td>0.340</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>765</th>\n",
       "      <td>5</td>\n",
       "      <td>121</td>\n",
       "      <td>72</td>\n",
       "      <td>23</td>\n",
       "      <td>112</td>\n",
       "      <td>26.2</td>\n",
       "      <td>0.245</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>766</th>\n",
       "      <td>1</td>\n",
       "      <td>126</td>\n",
       "      <td>60</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.1</td>\n",
       "      <td>0.349</td>\n",
       "      <td>47</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>767</th>\n",
       "      <td>1</td>\n",
       "      <td>93</td>\n",
       "      <td>70</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>30.4</td>\n",
       "      <td>0.315</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Pregnancies  Glucose  BloodPressure  SkinThickness  Insulin   BMI  \\\n",
       "763           10      101             76             48      180  32.9   \n",
       "764            2      122             70             27        0  36.8   \n",
       "765            5      121             72             23      112  26.2   \n",
       "766            1      126             60              0        0  30.1   \n",
       "767            1       93             70             31        0  30.4   \n",
       "\n",
       "     DiabetesPedigreeFunction  Age  Outcome  \n",
       "763                     0.171   63        0  \n",
       "764                     0.340   27        0  \n",
       "765                     0.245   30        0  \n",
       "766                     0.349   47        1  \n",
       "767                     0.315   23        0  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#按照惯例也看一下最后，对比是否有不同。似乎没有什么不同\n",
    "train.tail()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "看到怀孕次数10次、8次，我就震惊了，一会儿看一下是不是因为统计错误产生的离群点。\n",
    "关于糖尿病家族史，应该是家族史上的患病概率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 768 entries, 0 to 767\n",
      "Data columns (total 9 columns):\n",
      "Pregnancies                 768 non-null int64\n",
      "Glucose                     768 non-null int64\n",
      "BloodPressure               768 non-null int64\n",
      "SkinThickness               768 non-null int64\n",
      "Insulin                     768 non-null int64\n",
      "BMI                         768 non-null float64\n",
      "DiabetesPedigreeFunction    768 non-null float64\n",
      "Age                         768 non-null int64\n",
      "Outcome                     768 non-null int64\n",
      "dtypes: float64(2), int64(7)\n",
      "memory usage: 54.1 KB\n"
     ]
    }
   ],
   "source": [
    "#查看数据的基本信息\n",
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pregnancies                 0\n",
       "Glucose                     0\n",
       "BloodPressure               0\n",
       "SkinThickness               0\n",
       "Insulin                     0\n",
       "BMI                         0\n",
       "DiabetesPedigreeFunction    0\n",
       "Age                         0\n",
       "Outcome                     0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看一下数据集中是否又空值\n",
    "train.isnull().sum()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "看来数据集很完整，没有发现空值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pregnancies</th>\n",
       "      <th>Glucose</th>\n",
       "      <th>BloodPressure</th>\n",
       "      <th>SkinThickness</th>\n",
       "      <th>Insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>DiabetesPedigreeFunction</th>\n",
       "      <th>Age</th>\n",
       "      <th>Outcome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3.845052</td>\n",
       "      <td>120.894531</td>\n",
       "      <td>69.105469</td>\n",
       "      <td>20.536458</td>\n",
       "      <td>79.799479</td>\n",
       "      <td>31.992578</td>\n",
       "      <td>0.471876</td>\n",
       "      <td>33.240885</td>\n",
       "      <td>0.348958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3.369578</td>\n",
       "      <td>31.972618</td>\n",
       "      <td>19.355807</td>\n",
       "      <td>15.952218</td>\n",
       "      <td>115.244002</td>\n",
       "      <td>7.884160</td>\n",
       "      <td>0.331329</td>\n",
       "      <td>11.760232</td>\n",
       "      <td>0.476951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.078000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>27.300000</td>\n",
       "      <td>0.243750</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>117.000000</td>\n",
       "      <td>72.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>30.500000</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0.372500</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>140.250000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>127.250000</td>\n",
       "      <td>36.600000</td>\n",
       "      <td>0.626250</td>\n",
       "      <td>41.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>17.000000</td>\n",
       "      <td>199.000000</td>\n",
       "      <td>122.000000</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>846.000000</td>\n",
       "      <td>67.100000</td>\n",
       "      <td>2.420000</td>\n",
       "      <td>81.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Pregnancies     Glucose  BloodPressure  SkinThickness     Insulin  \\\n",
       "count   768.000000  768.000000     768.000000     768.000000  768.000000   \n",
       "mean      3.845052  120.894531      69.105469      20.536458   79.799479   \n",
       "std       3.369578   31.972618      19.355807      15.952218  115.244002   \n",
       "min       0.000000    0.000000       0.000000       0.000000    0.000000   \n",
       "25%       1.000000   99.000000      62.000000       0.000000    0.000000   \n",
       "50%       3.000000  117.000000      72.000000      23.000000   30.500000   \n",
       "75%       6.000000  140.250000      80.000000      32.000000  127.250000   \n",
       "max      17.000000  199.000000     122.000000      99.000000  846.000000   \n",
       "\n",
       "              BMI  DiabetesPedigreeFunction         Age     Outcome  \n",
       "count  768.000000                768.000000  768.000000  768.000000  \n",
       "mean    31.992578                  0.471876   33.240885    0.348958  \n",
       "std      7.884160                  0.331329   11.760232    0.476951  \n",
       "min      0.000000                  0.078000   21.000000    0.000000  \n",
       "25%     27.300000                  0.243750   24.000000    0.000000  \n",
       "50%     32.000000                  0.372500   29.000000    0.000000  \n",
       "75%     36.600000                  0.626250   41.000000    1.000000  \n",
       "max     67.100000                  2.420000   81.000000    1.000000  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#探索数据\n",
    "#查看各数据的统计特性\n",
    "train.describe()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "Insulin数据中平均数和中位数差距略大，暂时不明白是什么原因"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEICAYAAAC55kg0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXuUJUd937+/mdlZsTtrPXYGEEg7gwhHCXACiAlPhwjvGksKL+eAI1mxCQbPYRZywDkkvBIMCetjsLEd2xyLtRE47FqIdzhCMsgYwgEb4ZWQxApJIGBXWklol6cAYQfBL390N1vbU9Vdr77dM/39nFNn7u2urvrVr359a+6tqm+LqoIQQsg4merbAEIIIf3BQYAQQkYMBwFCCBkxHAQIIWTEcBAghJARw0GAEEJGDAcBQgaCiLxWRP6ibzvIuBDuEyDrCRE5BOBBAH4C4IcArgTwn1T1B33aRch6hd8EyHrkWao6B+AcAP8KwH8zT0oBY5sQD3ijkHWLqt4J4CoAjxaRT4nIHhH5LID7AJwlIieLyDtE5G4RuVNE3iQi0wAgItMi8lYR+aaIfF1EXiYiKiIz5flPicj/FJHPisj3ReTjIjJf1S0i7xORb4jI90Tk0yLyKOPcu0TkbSLy0fLaa0Tk4cb5R4nI1SLybRG5R0ReWx5/g4jsM/I9SUT+TkS+KyI3iMi5HbuUjBAOAmTdIiJnArgAwBfKQ78GYAXANgCHAfwlgPsB/DMAjwPwDAAvLvP+JoDzATwWxTeK51qq+FUALwTwQACzAF5pnLsKwCPKc9cB2F+79iIAbwRwKoDbAOwpbd4G4G8A/DWAh5S2fcLStocC+CiANwE4raz7AyKy0OQTQkLhIEDWIx8Wke8C+AyA/wvgd8rj71LVm1T1fhQfnOcDeIWq/lBVjwL4QwAXlnl/BcD/UtUjqvodAL9rqeedqvplVf0RgPeiGDAAAKp6qap+X1X/CcAbADxGRE42rv2gqn6+tGW/ce0zAXxDVd+qqv9YlnGNpe7/AOBKVb1SVX+qqlcDOIBi0CMkGzN9G0BIBM9V1b8xD4gIANxhHFoEsAnA3eU5oPinp8rzkFp+83XFN4zX9wGYK+uaRvGf/fMBLAD4aZlnHsD3mq4FcCaArzpbdqL9zxeRZxnHNgH4pMe1hHjDQYBsJMylbncA+CcA8+V/43XuBnCG8f7MgHp+FcBzAOwCcAjAyQC+A0AarjHtusgz37tV9TcD7CIkGP4cRDYkqno3gI8DeKuI/JyITInIw0Xk35RZ3gvg5SLyUBE5BcCrAorfhmKA+RaALTj+c5QPVwB4sIi8QkQ2i8g2EXmiJd8+AM8SkV8qJ7FPEpFzReQMS15CouEgQDYyv45iQvdLKP5Tfz+A08tzf45ikLgRxcTylSgmkX/iUe7/RjHxfGdZ9ud8DVLV7wP4RQDPQvGT0VcAPN2S7w4U3zZeC+AYim8G/wW8Z0lmuFmMEAAicj6AS1R1sW9bCJkk/K+CjBIReYCIXCAiM+VyzN8G8KG+7SJk0vCbABklIrIFxfLSfw7gRyjW5L9cVe/t1TBCJgwHAUIIGTH8OYgQQkbMIPcJzM/P69LSUt9mEELIuuHaa6/9pqoGy4q0DgIicimKre5HVfXR5bHLAZxdZjkFwHdV9bGWaw8B+D6KZXf3q+qyj1FLS0s4cOCAVwMIIYQAInI45jqfbwLvAvCnKNZGAwBU9d8bFb8Vx7fK23i6qn4zxjhCCCHd0joIqOqnRWTJdk4KUZZfAfALec0ihBAyCVInhv81gHtU9SuO8wrg4yJyrYisJNZFCCEkM6kTwxcBuKzh/FNV9S4ReSCAq0XkFlX9tC1jOUisAMCOHTsSzSKEEOJD9DeB8glM/w7A5a48qnpX+fcoit2YT2jIu1dVl1V1eWGBz80ghJBJkPJz0C4At6jqEdtJEdlaPkUJIrIVxVOdDibURxzs3w8sLQFTU8Xf/fVnXBFCiIPWQUBELgPw9wDOFpEjIvKi8tSFqP0UJCIPEZEry7cPAvAZEbkBwOcBfFRV/zqf6QQoPvBXVoDDhwHV4u/KCgcCQogfg5SNWF5eVu4T8GNpqfjgr7O4CBw6NGlrCCF9ISLX+u7FMqFsxDrn9tvDjhNCiAkHgXWOayEVF1gRQnzgILDO2bMH2LLlxGNbthTHCSGkDQ4C65yLLwb27i3mAESKv3v3FscJIaSNQaqIkjAuvpgf+oSQOPhNgBBCRgwHAUIIGTEcBAghZMRwECCEkBHDQYAQQkYMBwFCCBkxHAQIIWTEcBAghJARw0GAEEJGDAcBQggZMRwECCFkxHAQIISQEcNBgBBCRgwHAUIIGTEcBAghZMS0DgIicqmIHBWRg8axN4jInSJyfZkucFx7nojcKiK3icircxpOCCEkHZ9vAu8CcJ7l+B+q6mPLdGX9pIhMA3gbgPMBPBLARSLyyBRj1wv79wNLS8DUVPF3//6+LSKEEDutg4CqfhrAtyPKfgKA21T1a6r6/wC8B8BzIspZV+zfD6ysAIcPA6rF35UVDgSEkGGSMifwMhG5sfy56FTL+YcCuMN4f6Q8tqF53euA++478dh99xXHCSFkaMQOAn8G4OEAHgvgbgBvteQRyzF1FSgiKyJyQEQOHDt2LNKs/rn99rDjhBDSJ1GDgKreo6o/UdWfAvhzFD/91DkC4Ezj/RkA7mooc6+qLqvq8sLCQoxZg2DHjrDjhBDSJ1GDgIicbrz9ZQAHLdn+AcAjRORhIjIL4EIAH4mpbz2xZw+wZcuJx7ZsKY4TQsjQ8FkiehmAvwdwtogcEZEXAXiLiHxRRG4E8HQAv1XmfYiIXAkAqno/gJcB+BiAmwG8V1Vv6qgdg+Hii4G9e4HFRUCk+Lt3b3GcEEKGhqg6f6bvjeXlZT1w4EDfZhBCyLpBRK5V1eXQ67hjmBBCRgwHAUIIGTEcBAghZMRwECCDZqwSHGNtN5k8M30bQIiLSoKj2oFdSXAAG3u11VjbTfqBq4PIYFlaKj4A6ywuAocOTdqayTHWdpM0uDqIbDjGKsEx1naTfuAgQAbLWCU4xtpu0g8cBMhgGasEx1jbTfqBgwAZLGOV4Bhru0k/cGKYEEI2AJwYJoQQEgwHAUIIGTEcBAghZMRwECCEkBEzykEghy6LqwxqvviT01cxZY29r9Zz+7uyvYtyB+9nVR1cevzjH69dsW+f6pYtqsDxtGVLcTy1jNXV9LLHQo5+SCkrZ/3rkfXc/q5s76LcSfoZwAGN+Lzt/QPflrocBBYXT+yQKi0uppcxPZ1e9ljI0Q8pZeWsfz2yntvfle1dlDtJP8cOAqPbJzA1VXRDHRHgpz9NK8NFSNljIUc/pJSVs/71yHpuf1e2d1HuJP3MfQKe5NBlceWdnk4veyzk1MeJKWvs+jzruf1d2d5FuevBz62DgIhcKiJHReSgcez3ROQWEblRRD4kIqc4rj0kIl8UketFZBBbgHPosrjKWFmh5osvOfVxYsoauz7Pem5/V7Z3Ue668HPb70UAngbgHAAHjWPPADBTvn4zgDc7rj0EYD70N6ou5wRUi0mZxUVVkeJv7GSkrYwcZY+FnL6KKWvsfbWe29+V7V2UOyk/o8s5ARFZAnCFqj7acu6XATxPVdfIW4nIIQDLqvrNkIGJ2kGEEBJGn3MCvwHgKsc5BfBxEblWRFYy1EUIISQjSc8YFpHXAbgfgGv7w1NV9S4ReSCAq0XkFlX9tKOsFQArALBjSLMmhBCygYn+JiAiLwDwTAAXq+M3JVW9q/x7FMCHADzBVZ6q7lXVZVVdXlhYiDWLEEJIAFGDgIicB+BVAJ6tqvc58mwVkW3VaxSTyQdteckwSd3u3ud2ed+6m/J1bX8f/hlLnUNi8O1vmzkGcBmAuwH8GMARAC8CcBuAOwBcX6ZLyrwPAXBl+fosADeU6SYAr/Odre56dRBpJ3W7e5+yBL51N+Xr2v4+/DOWOofEepCNGN2OYeLH0hJw+PDa44uLwKFD3V+fgm/dTfmAbu3vwz9jqXNITLL9sauDOAgQK6nb3fuUJfCtuykf0K39ffhnLHUOCcpGkHVL6nb3PrfL+9bdlK9r+/vwz1jqHBLrof0cBIiV1O3ufW6X9627KV/X9vfhn7HUOSTWRftjJhK6TpwYHgap2937lCXwrbspX9f29+GfsdQ5JDaEbMSk4ZwAIYSEwTkBQgghwXAQIISQEcNBgBBCRgwHAUIIGTGjGgRsGh6D1/XoiEm2e6w+3ujs3w/Mzxcbn0SK12196xMLsfFSv2737n7jbt3EfcySoq5TF0tEbRoemzapzs6OT9dkknomY9eO2ajs21fcP2a/AsX91LQcty0WYuPFdl09TTLu+oh7cIloMy4NDxsbXddkknomY9eO2ag03U+uvvWJhdh48b2/JxV3fcQ9tYNacGl42NjouiaT1DMZu3bMRqXpfnL1rU8sxMaL7/09qbjrI+65T6CFEK2OIel6dMEk9UzWg3YKCaep/0L73DweGy+pmla5WU9xP5pBwKbhsWkTMDt74rHB6Xp0wCT1TNaFdgoJZs+e4v6pMzvr7lufWIiNF9t1dSYZd+sq7mMmErpOXWkH2TQ8xqprMsl2j9XHG519+1S3bz8+8bl9u98EblssxMZL/brV1X7jbtJxD04ME0LIeOGcACGEkGA4CBBCyIjhIEAIISOGgwAhhIwYr0FARC4VkaMictA4dpqIXC0iXyn/nuq49gVlnq+IyAtyGd5EXdPE1DUx9Tzm54s0eG2PkqFrkfjaF6vx0qXuzFCp2iMCzMwUf2PaleKXXFo+ufoipFyb/2L8mKuc1PZ0gs8SIgBPA3AOgIPGsbcAeHX5+tUA3my57jQAXyv/nlq+PrWtvpQloi5NE0B1enqtVtB60bQZugaPr32xGi9d6s4MlSZfhbQrxS85tXxy9EVIubn0hLrUJcrpJ0QuEfXPCCzVBoFbAZxevj4dwK2Way4C8Hbj/dsBXNRWV8ogsLjY3FltaXExuupOcbVrKPb62ufbP77XmfmG7qNQ2nzl264Uv8Re21VfhJQbG2tdlZPanjZiBwHvfQIisgTgClV9dPn+u6p6inH+O6p6au2aVwI4SVXfVL7/7wB+pKq/byl/BcAKAOzYsePxh33V3mqEaATZGKqmzdA1eHzti9V46VJ3Zqi0+cq3XSl+ya3lk9oXIeXm0hPqUpcop5+Guk9ALMes7lTVvaq6rKrLCwsL0RWmanMMUdsDGL4Wia99sRovXerODJWu9XJ8rs+t5dPV/Wk7nktPqEtdoiHEbMogcI+InA4A5d+jljxHAJxpvD8DwF0Jdbbi0jQBgOnptVpBJoPV9sDwtUh87YvVeOlSd2aoNPkqpF0pfsmp5ZOjL0LKzaUn1KUu0SBi1vd3I6ydE/g9nDgx/BbLNacB+DqKSeFTy9entdWVqh1U1zQBjuuamHoe27cXab1o2gxdg8fXvliNly51Z4ZK1R6gWNhQ/V4c2q4Uv+TS8snVFyHl2vwX48dc5aS2pwl0OScgIpcBOBfAPIB7APw2gA8DeC+AHQBuB/B8Vf22iCwDeImqvri89jcAvLYsao+qvrOtPmoHEUJIGHyoDCGEjJihTgwTQggZMBwECCFkxIxqELBtzx6qjIRrK/kkZQRC88zPA3NzdrmOvmhqQ87t+l1t/d+9+0SZgt2708vsUgIhJ7l8amtvzL3ehXxGF/0bTMxsctepiyeL2bZnb9o0TBkJ11by1dXJyQjE5rGl2dlh+bFaJZZru35XEgmrq3Z/rq7Gl9mlBEJOcvnUN0bbyu5CPiN3/6Jr2YhJpi4GgVg5iT4kB1y2VkvSUm3NJcEQ4tMh+XFxMe92/a4kElz9PT0dX2aXEgg5yeXTXDHahXxG7v6NHQRGszooVk6iD8mBrm3NJcEQYueQ/CjlPva29uWoJ6XNlZ02Ym/bLiUQcpLLp7litAv5jCa74u5/rg5qJHYbdh+SA646p6fjrvPNFyrBEOKbIflxx4682/W72vrv6u+2OGiiSwmEnOTyaa4Y7UI+o4v+jSLm60PXiXMCnBPIAecE/GwdSty32ck5gWbAOYF2bNuzhyoj4dpKPkkZgdA827erbt16YkBXch190dSGnLIGXUkkrK6eKFOQMgBUdCmBkJNcPrW1N+Ze70I+I2f/xg4Co5kTIISQjQznBAghhATDQYAQQkYMBwFCCBkxHAQIIWTEbKhBoElXJ0b3o0tdoRzaPTHnTa2SqSlg2za3ltIQdGRS+81s1/x8mqZRSp+l9KUrDm0xmRozrvak3FP795+oKTU9PTmNnDa9sNx6Uj7aXkO7x3pfDmpLMUtEm9YD2/YDxKwLzrWWOsc6/ZjzU1Pu9sT6qUti1ma79oPYtuiH7F9I6TPb/g7fvvRd5+7Tf6H+zHFP7dvnjrscy12b8N0blGvvSJO/utifUgdj3ycQow0UoxXie32MrSHaPbHnY1JfOjIxei2h7Z6E3pJLI8anL3P0o2/MpPrSVlZTGSkaSD6E6gal6hW11Zdbs6pO7CCwYfYJxOjtxGiF+F7fRA7tntjzMfSlIxOj1xLa7knoLbXVHaNxFIJvzNTJcU+1ldHlx0+obpDLntQYyVlHE6PfJ5BT88W3vNx6RCHaPbHnY+hLRyZGr6UrbZmUPnNpwfj0ZQ7f+8aM73GfunzK6FojJ1Q3KNUPPjpCXelMJRHz9aHrxDkBzgn4tNH3Gs4JxPmTcwKcE2i+EDgbwPVGuhfAK2p5zgXwPSPP633KjtUOatLVidH96FJXKId2T8x5U6tERHVuzq2lNAQdmdR+M9u1ffvxGy9G0yilz1L60hWHtphMjRlXe1LuqX37TtSUmprqfgCo2+/ro9T4b/JXrjpcxA4CWeYERGQawJ0Anqiqh43j5wJ4pao+M6Q8agcRQkgYfc8J7ATwVXMAIIQQMnxyDQIXArjMce7JInKDiFwlIo9yFSAiKyJyQEQOHDt2LJNZhBBCmkgeBERkFsCzAbzPcvo6AIuq+hgAfwLgw65yVHWvqi6r6vLCwkKqWYQQQjzI8U3gfADXqeo99ROqeq+q/qB8fSWATSIyn6FOb+pyAWYypQN27y7W+dryVWluzr1Nv75NfPfudFkIn7aFyFr45rdJLOzadbx9MzNF+3Lab8oKiBRyFk3lhUgWDGqLfgZ8pAl88qX4Jqdf62U13TtdSrmYdbRJjOSM/d7jMmY22UwA3gPghY5zDwZ+Nvn8BAC3V++bUq4ni+3bVywJa1pWNzurunNnc56QJXmuFPP4xra2hSxh9c3v47Mqzcx0Z39VvmsFjo/vulyO1xc+yxB98rUtWw21IeejOUOWV+buW1f8m8uJu7h3c9iOPmQjAGwB8C0AJxvHXgLgJeXrlwG4CcANAD4H4Ck+5eYaBHJKJ+RIIbIQOdrmU189f6rPctrvKs/Xd11u0e8LH2kCn3xtUhYxNsT4NSQOupRy8bEnVn7Dt45U22MHgQ0jG2Ejp3RCDkJkIdoIlbXwzQ+k+Syn/a7yfH2X6uMh4iNNkCIb4uObnH4NiQMgLOZjaLInVn7Dt45U2/teIjpIet2KbSFEFiKkrJT66vlTfZbTflc+X98Ncot+Ij7SBD752qQsYmzILd1Sz9ellIvP9bHyG775eovLmK8PXSfOCfi1jXMCzXVzTsCdj3MCbns4JzCAlGsQUF0rF2AmUzpgdbXYxt0UjFu3urfpAyduE19dTZeF8GlbiKyFb36bxMLOncfbNz1dtC+n/aasAFDIWTSVFyJZMCQZjBz4SBP45EvxTU6/1stqundCYz7WnjaJkZyxn8v22EFgQ88JEELIWOCcACGEkGA4CBBCyIjhIEAIISOGgwAhhIyYDTMIxGhxdK3f4avxkssWsz6XDpJNB6XJhpRjqbjKbNKP8dFsCq0v5LpQPZvQOkM1k+qxUO9/m05Omw/r7TV1n6anj5dTlVvFvukb23kzlkybbJpdIT5x+aKy1dePbfdwGyEaSRMlZklR1yl0iWjMutuu15D7rufOZYvPemvbmuem6237IHyPpfrS5RPb+nafNeax9bVdm7J2PbTOlP0Rtv733RPiu/8gR3I9CjTEnhh7Q/3Yxf2Zes9gzPsEYrQ4utaV8dV4yWVLqOaPr85OSkrxpcuetg+IWDti+yBFzya0zlTNpPo1IX3uq0k0qeSr4xPTRl8/dnF/ptwzsYPAhtgnEKPF0bWujK/GSy5bQrVifHV2UkjxZU57utTDCdVwSqkzVTOpfg3g7+NUTaLc+Or4hNgb6scu7s+Ue2bU+wRitDi61u/w1XjJZUuo3b46OymklOm61qV5k2pHbB90UXbq8S50onw1iSaFr45Pyj2Uo+9D8/bi35ivD10nzglwToBzAvH5OSfAOYGQFHzBJFKMdlCMFkfXujK+Gi+5bDHrc+kg2XRQmmxIOZaKq8wm/RgfzabQ+kKuC9WzCa0zVDOpHgv1/rfp5LT5sN5eU/dpaup4OVW5VeybvrGdN2PJtMmm2RXiE5cvKlt9/dh2D7cRopEUQ+wgsCHmBAghZOyMek6AEEJIHBwECCFkxHAQIISQEcNBgBBCRkzyICAih0TkiyJyvYismc2Vgj8WkdtE5EYROSe1ziZ8dFxseilteiSVzoepIVL/G6obAxTlVtdPTQHbtrXb26alUm+Hea15fV0PZffutX7ZtetEvZVt2+I1Tto0h+o6NGa/mH6amSneh9Rp66/Khl271mosuTRo6r6stHba4snXH67zMbGVU8+pa52tUBua/NGVHpPLhlRdod59G7OkyEwADgGYbzh/AYCrAAiAJwG4pq3M2MdL+qzZXl2Ne4ZoTGpb97u62n69y16fenzXgYcm17N/Q/vG5xnNTc+AXl1Ni4eQ/gv1pW0vRpttIevbJ7kPpus9NbE22Ozpau+Fjw0xvsnpW/S1T8BjEHg7gIuM97cCOL2pzNhBwEefo2kjShf6KE1aID46OL5aObZ6utR5CdU46cKW6enu60yJiRTtqhRNIp/yQ+haZyvFhro9Xekx+doQ6pucvo0dBJL3CYjI1wF8B4ACeLuq7q2dvwLA76rqZ8r3nwDwKlU9UMu3AmAFAHbs2PH4w4cPB9uSqmvShT5KkxZIpd/SVT1d6ryEapx0ZUtTmTnqTImJFO2qFE0in/JD6FpnK8WGuj1AmK0hbUvRIXKR07d97hN4qqqeA+B8AC8VkafVbbNcs6bZqrpXVZdVdXlhYSHKEB/djSbtmS70UZrK8tHB8dXKsdXTpQ5JqlZRDtp8k6POlJhI0a5K0STyKT+ErnW2Umyo5+lSp2mSWkET1RCK+frgSgDeAOCVtWMT+zmIcwJr/cE5gTz9xzmBPGXFwjmBdtDHnACArQC2Ga//DsB5tTz/FidODH++rdzYQUDVT8fFppfSpkdS6XwAx3+nr/8N1Y1RLcqtrhdRnZtrt7dNS6XeDvNa83rT9qqNdb/s3Hmi3srcXPzN36Y5VNehMfvF9NP0dPsAUK/T1l+VDbZBxqVBU/dlpbXTFk++/nCdj4mtnHpOXWhDpdjQ5I+u9JhcNthiKrZdKb6NHQSS5gRE5CwAHyrfzgD4K1XdIyIvKb9lXCIiAuBPAZwH4D4AL9TafEAdagcRQkgYsXMCMymVqurXADzGcvwS47UCeGlKPYQQQrqBO4YJIWTEcBAghJARs2EHAd+t2D4SEq6ym7aLu8o17ZqbK5Y5mjIEMzOFjIHNdte2efO1WWZdXsEl2xDSfpt0gsjxOl2+9q0nZgt96DU2CQqXLIWrbJvch02Gw1fmICUGTVkPWzz5Smy4fGrKI5h11e2MlT/wua7pfkqNK58+dsmJ2Oo3+911//YlvWElZja565SyOkjVf9mVa9lf0/K+pmVi5jI1W7nT0+1LIl3lhj5W0Uyrq+4lmrYdya72+y6TtD0+08fPMcvlQq9pW5Zrpp077WW7lqyG9mlTrMTGYFMM+BJSvvmoylyP5/SNn+lpv/htqsN1rqmP2/rO9/7NvcwWY3+8pInvVuymbeCh28zN67qSSEi5NtSmnNvmff0cs4U+9JoUP+ZObbESG4OuGPAlJlZi5Q98rkuV7GiqI/ZeDb3WFXc5pTdiB4EN+XhJ363YTdvAQ7eZm9cBzXn6QCTMppzb5n39HLOFPvSanFIdqbTFSmwMuvC9JrT8GLmGtrp846fJJp+4ctntU37stfVycklv8PGSBr5bsbvYdt+0dT0FX/kI17U5ZB5it837+jnG56HXpPgxN22xklNSIKTdMbGScr+0HU+VumiqI/ZeDb3W5f+JykO4iPn60HXinMDacjknkNbXFZwTaIdzApwT6D2lDgKq/luxfSQkXGVXgQjYt67byjXt2rpVdWpqbWDv3Gm33bVt3nxtllmXV3DJNoS03yadAByv0+Vr33pittCHXmOToHDJUrjKtsl91OOhkhrxkTlIiUFT1sMWTyEDgM2npjyCWVfdzlj5A5/rmu6n1Ljy6eMq+fSd2e+u+7cL6Y3YQWBDzgkQQsjY4JwAIYSQYDgIEELIiOEgQAghI4aDACGEjJgNNwi49Dnqx3ftOlErxqXXk1qvDZdOjVmOqUPTpDtjs6NNS8WlQeTTbh/9IlOXqGqLmbZtc+ur+NRb6fLYNIy2bfPXibL5qM1/9fZv3nyibk9dc6iuJ1TXj0rVkPGJ9/n5QlPKbJMZ/1NTazVyKj/WY7XtPmnyn4/mlo9eT4wPbMeb7rEm/Z96DNl0wWx9beuHQegHxSwp6jrlfLxk7Br7rh4R51qn7lrLH2Kbz5rztjpCH8XnSj6PjuyiXsD++Mu2Nd2zs+2PHvW1w7a/oMkvuR/7mLKnJDT5PIKz8m3b/pqmPRwhj3xsWpcf65u2R73a9gD5Pj4111JRcJ+AW8sjVi/GV9cjRDfFZUuovk+otk+V36eOnJo1Ofydqh3jW0ZTnHSlCRUaaz5tmrQ+kk9stdm0uOiXJ9UHqRpcQ+l7G7GDwIbaJxCrqeLCV9cjRL+mSbsmRN8nVNunyu/jo9yaNb7krDdEJ8q3PKC79sdoyHTdH76ExFZTGTFxORQfpJBLP4j7BJBfL8ZX1yNEN8VlS6i+T6i/rvyVAAAQgklEQVS2T3XOp46cmjUh5Kw3RCeqoilOutKEMsvPdc2k9ZF8YqvNph07/PL4HGuqL1WDqwt61w+K+frQdeKcQJxtnBMoEucEOCfAOYEJzAkAOBPAJwHcDOAmAC+35DkXwPcAXF+m1/uUnaId5NLnqB/fufNErRiXXk9qvTZcOjVmOdU5oFl3xmZHm5aKS4PIp90++kWmLpHtN9u5Obe+ik+9lS6PTcNobs5fJ8rmozb/1dtv3uRTU2s1h8x+rPslh4aMT7xv315oSpltMuNfZO3v3ZUf67Hadp80+c9Hc8tHryfGB7bjTfdYk/5PPYZsumC2vrb1Q079oNhBIHpOQEROB3C6ql4nItsAXAvguar6JSPPuQBeqarPDCmb2kGEEBLGxOcEVPVuVb2ufP19FN8IHhpbHiGEkMmTZWJYRJYAPA7ANZbTTxaRG0TkKhF5VEMZKyJyQEQOHDt2LIdZhBBCWkgeBERkDsAHALxCVe+tnb4OwKKqPgbAnwD4sKscVd2rqsuqurywsJBqFiGEEA+SBgER2YRiANivqh+sn1fVe1X1B+XrKwFsEpH5lDoJIYTkI3oQEBEB8A4AN6vqHzjyPLjMBxF5Qlnft2Lr9CVUm8VHS8emGTI9fVwDxNQEqdLUFHDSScffz825dXpsNps6KlNThZ5L9boqc/Pm43aYWit1bZ02nZImH/jq/JiaLKaNlX1VWS5fmbbW7a/KMPOZqfKtrawYfZYmfSdXn/nqzVRtadKtMX1kahI11V9p1dj0keo+CtXMacPV93WNnZC43L9/bayYulOudjRpOtXLb4uxuTl7vNrs3r3b3XabL320lCZCzJKickXRzwNQADfi+BLQCwC8BMBLyjwvQ7F89AYAnwPwFJ+yU5eIhjzL02fdfNP64JTU9JzT+mMCc6SmZweHrJ12Pa95UmvTc7XbRdNejpD25o6dtvpdz4zOEaNNpPa9rX/27ct/D9T9l9ovpt0+z66u76lo2zcRCqgdVBCi49OU30xdarF0rUnj44dJafP0mUL0WZp0Z0LbmzN2YuqfhO9y2DKJeOrCf5Xdvv3sq+EVoycUOwhsKO0gIEzHpyn/pOhak8ZWXw79ldzaPF0Tos9S9YmNqo19tbeP+tt8l8OWScVTbv9VdjfFjC1/W/0xekLUDioJ1Z9J1ZRJpWtNGlt9PsdCy+ld/6SFEPt8dGd8y8sZOzH156Ctrhy2TCKeuvBfVY5vP/tqeE30for5+tB14pwA5wQm0W4XnBNo7uvU+PHpH84J+MdrBTgncJxQbRYfLR2bZkgVpHVNkCqJqG7efPz91q1unR6bzaaOikih51K9NgOxfrPYtHXadEqafOCr82Nqspg2VvZVZbl8Zdpat78qw8xnpsq3trJibqgmfSdXn/nqzVRtadKtMX1kahI11V/9Q2HTR6r7KFQzpw1X35vJ1q9N/bNv39pYMXWnXO1o0nSql98WY1u32uPVZvfqqrvtLp2lNi2lEGIHgQ03J0AIIWOEcwKEEEKC4SBACCEjhoMAIYSMmA0zCNS3gFdb1Ktt2VWamQF27XJvjTe3cru2fru2yFdbyZu27tukEpq2/Nu2p7fVb+apywHUt6jbjru2sNtkLJpkJkLsqx+P2Upv2tckX+CSAzBjxmxXUx/WJSVsUgdmMmVE6nIFdfkRH5mHNukRW9y0nbf1W13uoald8/PFPWbGk+2e27Wr+d6wST24YsSnr+pyHfW4s0l1NPnM1dchnwOVTXNzJ9rnansnxMwmd51CVwelLvdqWqpZT22PjJuejlum17S8L+TxkG2P8ov1jc8SOJ/ksq/t0X9ty+Zy2RebqseDdrG016d/2pZGx5wfQqqv6sltZxV3rn6bmXH7u6mvYz8HmtreBsa8RDTXtvUhSx/4bjevAjB33bnlD2LsbtpK36W0h2+b+oqfpthti5uQuOrLr7nvdVsdufzdZdvbiB0ENsQS0Vzb1oH0crpCxG+7eVd1D8EvlQ9c5/qmLz81xW5b3PQZV76Ydg3Bzkl+VoTUMeolorm2rQ9Z+sB3uzmQX+Zix47u5A98jpt2hJY5Kaan+4ufpthti5uQuOqDer92LSfhw6Q+KyYV0xtiENizB9i0Kf76LVuKMvbsKV43sWkTMDvrPj89Hdd5mza5r5udLWwD2m2cnQVWVtrb4Uvlm5WVPOW57Nuypdnuyg4XueyLZWWlsG9qwndUU+yaPos5PwTq/ZrbziruXP02M7M2f+Xvpr6O/RwwmVhMx/yG1HWKkY2obwGvtqjXf7ubnlbdudO9Nd7cym37LbCaFLJtka+2kjdt3TdTdW3Tln/b9vS2+s08dTkA8zdQ13HXFnabjEWTzESIffXjTXa4MO2r111PNjkAM2bMdrX9blvXpLHJDFTJlBGpkkt+xEfmoU16xBY3bedt/VaXe2hq1/btxT1mxpPtntu5s/necE2M2mLEp6/qch31uLNJdTT5zNXXIZ8DlU1bt55oX+iksKoqxjwnQAghY2fUcwKEEELi4CBACCEjhoMAIYSMGA4ChBAyYpIGARE5T0RuFZHbROTVlvObReTy8vw1IrKUUl8bNp2PNr2UtjJ2716rD1Jp5tTraNK7Mctt0tqp6/fU667rDZnl17WTQmyrX2tqv1Q6QTb7fPRuTD/Z9J1sdbt0fzZvXtsXrn42tYR89FiadKPqfnXFiq2PTV0bs111LRtbm832uTSJTL2k+nWu9rn0c1xaN5WWTpPPbNpTpk/qsW7ab9PpcWlQ1eOnuiZEf8fW1+Y1bZpBbZ8z9Xb73Ku9EbOkqFxRNA3gqwDOAjAL4AYAj6zl2Q3gkvL1hQAu9yk7domo7XF7dZ2flEdN2lKTllCIJlFMMsu3aSdNT/vZ1sWjM21+6kraQWRtO126Lk3LDn37KLc+k0/7mpa7utLMjP/jDH0fgxn6iM1Qe3P7zvVEsSa7d+506yz5fs7YfNx0v4U+AtUGJq0dBODJAD5mvH8NgNfU8nwMwJPL1zMAvgkUy1KbUswgEKLl4dKg6UIPpGudkZTyh66X1EVy6bGE+qFvraKQPvZpn297TP+th9ix9Xdf90vb9U3aWD7EDgLR+wRE5HkAzlPVF5fvfw3AE1X1ZUaeg2WeI+X7r5Z5vmkpbwXACgDs2LHj8YcPHw6yJ0RTRMSuQdOFLolI8Td3uTnK79q2oWJr7xA0abqgivWc7avKWS8+q9sYa3fq/dJ2vetzyb/8ye8TEMuxevN88hQHVfeq6rKqLi8sLAQbE6Ll0aajkpOudUZSyh+6XlIXuLbyh/qhb60iX3y1gXzbY+ZbD7Fja1df90vb9X35M2UQOALgTOP9GQDucuURkRkAJwP4dkKdTmyaIjadnyYNmhhdkiYtoRBNohjM8m3aSdPTfral6C750qSNlIrI2na6dF1ceiwhfZRbn6mNagIxlJkZP22gSj/HJw5M/+WO67pOTw5s/d1m986dbp0l38+ZOm33m6kPNnFifkMqf0KaAfA1AA/D8YnhR9XyvBQnTgy/16fsmDkBVbvOR5teSlsZq6tr9UEqzRyblpBL78Yst0lrp67fU6+7rjdkll/XTgqxrX6tqf1S6QTZ7PPRu3FpI9k0aaq6XROh9Qm4uTl3P5taQj56LE26UXW/umLF1semro3ZrrqWja3NZvtcmkS23/Or61ztc+nnuLRuKi2dJp/ZtKdMn9Rj3bTfptPj0qCqx091TYj+jq2vzWvaNIPaPmfq7fa5V1NBH9pBInIBgD9CsVLoUlXdIyL/ozTmIyJyEoB3A3gcim8AF6rq19rKpXYQIYSEETsnkPQFTFWvBHBl7djrjdf/COD5KXUQQgjpDu4YJoSQEcNBgBBCRgwHAUIIGTEcBAghZMQM8sliInIMQNiW4ePMo5CnGCJDtg2gfakM2b4h2wbQvhQq2xZVNXin7SAHgRRE5EDMMqlJMGTbANqXypDtG7JtAO1LIdU2/hxECCEjhoMAIYSMmI04COzt24AGhmwbQPtSGbJ9Q7YNoH0pJNm24eYECCGE+LMRvwkQQgjxhIMAIYSMmA0zCLQ99H5CNlwqIkfLJ6pVx04TkatF5Cvl31PL4yIif1zae6OInNOxbWeKyCdF5GYRuUlEXj4w+04Skc+LyA2lfW8sjz9MRK4p7btcRGbL45vL97eV55e6tM+wc1pEviAiVwzNPhE5JCJfFJHrReRAeWwo/XuKiLxfRG4pY/DJA7Lt7NJnVbpXRF4xIPt+q7wnDorIZeW9ki/uYvSnh5bg8dD7CdnxNADnADhoHHsLgFeXr18N4M3l6wsAXIXi6WtPAnBNx7adDuCc8vU2AF8G8MgB2ScA5srXmwBcU9b7XhQS5ABwCYDV8vVunPisissn1Mf/GcBfAbiifD8Y+wAcAjBfOzaU/v1LAC8uX88COGUottXsnAbwDQCLQ7APwEMBfB3AA4x4+485424ijp1Ax7U+9H6CtizhxEHgVgCnl69PB3Br+frtAC6y5ZuQnf8HwC8O0T4AWwBcB+CJKHZCztT7GcDHADy5fD1T5pOO7ToDwCcA/AKAK8oPgSHZdwhrB4He+xfAz5UfZDI02yy2PgPAZ4diH4pB4A4Ap5VxdAWAX8oZdxvl56DKURVHymND4EGqejcAlH8fWB7vzebyK+LjUPy3PRj7yp9argdwFMDVKL7dfVdV77fY8DP7yvPfA7C9S/tQPEDpvwKoHge+fWD2KYCPi8i1IlI9WHEI/XsWgGMA3ln+lPYXIrJ1ILbVuRDAZeXr3u1T1TsB/D6A2wHcjSKOrkXGuNsog4D3A+0HRC82i8gcgA8AeIWq3tuU1XKsU/tU9Seq+lgU/3E/AcC/aLBhovaJyDMBHFXVa83DDTb00b9PVdVzAJwP4KUi8rSGvJO0bwbFz6R/pqqPA/BDFD+vuOjr3pgF8GwA72vLajnWiX3lPMRzUDzG9yEAtqLoX1f9wbZtlEHA56H3fXGPiJwOAOXfo+XxidssIptQDAD7VfWDQ7OvQlW/C+BTKH5vPUVEqifgmTb8zL7y/MkoHmHaFU8F8GwROQTgPSh+EvqjAdkHVb2r/HsUwIdQDKRD6N8jAI6o6jXl+/ejGBSGYJvJ+QCuU9V7yvdDsG8XgK+r6jFV/TGADwJ4CjLG3UYZBP4BwCPKGfNZFF/pPtKzTRUfAfCC8vULUPwWXx3/9XKlwZMAfK/66tkFIiIA3gHgZlX9gwHatyAip5SvH4Ai+G8G8EkAz3PYV9n9PAB/q+UPoV2gqq9R1TNUdQlFfP2tql48FPtEZKuIbKteo/ht+yAG0L+q+g0Ad4jI2eWhnQC+NATbalyE4z8FVXb0bd/tAJ4kIlvKe7jyXb64m8RkyyQSihn7L6P4Hfl1PdlwGYrf7X6MYkR+EYrf4z4B4Cvl39PKvALgbaW9XwSw3LFtP4/ia+GNAK4v0wUDsu9fAvhCad9BAK8vj58F4PMAbkPxNX1zefyk8v1t5fmzJtjP5+L46qBB2FfacUOZbqrugQH172MBHCj798MATh2KbWWdWwB8C8DJxrFB2AfgjQBuKe+LdwPYnDPuKBtBCCEjZqP8HEQIISQCDgKEEDJiOAgQQsiI4SBACCEjhoMAIYSMGA4ChBAyYjgIEELIiPn/j1WLycs4FeUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x23ebce8f080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 单个特征散点图\n",
    "fig = pyplot.figure()\n",
    "pyplot.scatter(range(train.shape[0]), train[\"Pregnancies\"].values,color='blue')\n",
    "pyplot.title(\"Pregnancie\");"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "我们看一下刚才关注的怀孕次数的散点图，数据集中在15次以下（不包括15次）\n",
    "发现我对该数据的判断还是太天真了，不过考虑到印第安人还是可以理解的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnX2sJ9V537/P3t3Fvhebl8vGJcDeBQU5RVFjw22CX+RWXjsxjmWsym2xbmQSEa2ySVUSKiUgpKr5Y6XatWps1XKyMY6w99YmpW5BJKmDgPzRSsW5a2MMxpiNDcsGbNa8pfVGNZinf8wZ7+zszHmbMzNnfvP9SKP7+82d38wz5+V7nvOcM2dEVUEIIWQ+bBvbAEIIIcNC4SeEkJlB4SeEkJlB4SeEkJlB4SeEkJlB4SeEkJlB4SckAhH5KxH5DfN5Q0T+cmybCPGFwk8WEhF5QkTeNcS1VHVTVX9piGsRkgIKPyGEzAwKP1loROTXROR/isjHROQFEfmuiFxV+/93ROT/mP9tmP3/TkQOVY7bIyIqItvbrlH5riLymyLyuLnmp0RE+r5XQnyh8JM58IsAHgNwHoCPArhVClYAfBLAVar6OgBvBfBgomu+D8A/BvDzAP4FgF9OdF5COkPhJ3PgSVX9Y1X9MYDbAJwP4A3mf68C+DkRea2qPqOqjyS65r9X1RdV9SiA+wG8KdF5CekMhZ/Mge+VH1T1hPl4pqr+EMC/BPCbAJ4RkT8TkZ9NfU0AJwCcmei8hHSGwk9mjap+WVXfjaIX8C0Af2z+9UMAy5VD/8HQthHSFxR+MltE5A0i8n4T6/9/AP4vgB+bfz8I4B0isltEzgJw01h2EpIaCj+ZM9sA/BsATwN4HsA/AfBbAKCq9wC4HcBDAA4DuHskGwlJjvBFLIQQMi/o8RNCyMyg8BNCyMyg8BNCyMyg8BNCyMw4bd2RMTjvvPN0z549Y5tBCCGT4vDhwz9Q1V2hv8tC+Pfs2YOtra2xzSCEkEkhIk/G/I6hHkIImRkUfkIImRkUfkIImRkUfkIImRkUfkIImRkLJfybm8CePcC2bcXfzc2xLSKEkPzIYjpnCjY3gX37gBPmNRtPPll8B4CNjfHsIoSQ3FgYj//mm0+KfsmJE8V+QgghJ1kY4T96NGw/IYTMlYUR/t27w/YTQshcWRjhP3AAWF4+dd/ycrGfEELISRZG+Dc2gIMHgbU1QKT4e/AgB3YJIaTOwszqAQqRp9ATQoidhfH4CSGE+EHhJ4SQmUHhJ4SQmeEl/CLyuyLyiIg8LCJfEJHXiMjFIvKAiDwuIreLyE5z7Bnm+xHz/z193gAhhJAwnMIvIhcA+NcA1lX15wAsAbgGwEcAfFxVLwXwAoDrzE+uA/CCqv4MgI+b4wghhGSCb6hnO4DXish2AMsAngHwTgB3mP/fBuAD5vPV5jvM//eKiKQxlxBCSFecwq+qfwvgYwCOohD8lwAcBvCiqr5iDjsG4ALz+QIAT5nfvmKOX62fV0T2iciWiGwdP368630QQgjxxCfUcw4KL/5iAD8NYAXAVQ2HavkTy/9O7lA9qKrrqrq+a1fwS+IJIYRE4hPqeReA76rqcVV9GcCXALwVwNkm9AMAFwJ42nw+BuAiADD/PwvA80mtJoQQEo2P8B8FcKWILJtY/V4A3wRwP4APmmOuBXCn+XyX+Q7z//tU9TSPnxBCyDj4xPgfQDFI+1UA3zC/OQjg9wHcICJHUMTwbzU/uRXAqtl/A4Abe7CbEEJIJJKDM76+vq5bW1tjm0EIIZNCRA6r6nro7/jkLiGEzAwKPyGEzAwKPyGEzAwKPyGEzAwKP3GyuQns2QNs21b83dwc2yJCSBcW6g1cJD2bm8C+fcCJE8X3J58svgN82xkhU4UeP7Fy880nRb/kxIliPyFkmlD4iZWjR8P2E0Lyh8JPrOzeHbafEJI/FH5i5cABYHn51H3Ly8V+Qsg0ofATKxsbwMGDwNoaIFL8PXiQA7uETBnO6iFONjYo9IQsEvT4CSFkZlD4CSFkZlD4CSFkZlD4CSFkZlD4CSFkZlD4CSFkZlD4CSFkZlD4SSe4ZDMh04MPcJFouGQzIdOEHj+Jhks2EzJNKPwkGi7ZTMg0ofCTaLhkMyHThMJPouGSzYRMEwo/iYZLNhMyTTirh3SCSzYTMj3o8RNCyMyg8BNCyMyg8BNCyMyg8BNCyMyg8BNCyMyg8BNCyMyg8BNCyMyg8BNCyMyg8BNCyMyg8BNCyMyg8BNCyMzwEn4ROVtE7hCRb4nIoyLyFhE5V0TuEZHHzd9zzLEiIp8UkSMi8pCIXN7vLRBCcoOv5MwbX4//EwD+h6r+LICfB/AogBsB3KuqlwK413wHgKsAXGq2fQA+ndRiQkjWlK/kfPJJQPXkKzkp/vngFH4ReT2AdwC4FQBU9Ueq+iKAqwHcZg67DcAHzOerAXxOC/43gLNF5PzklhNCsoSv5MwfH4//EgDHAfyJiHxNRD4jIisA3qCqzwCA+ftT5vgLADxV+f0xs+8URGSfiGyJyNbx48c73QQhJB/4Ss788RH+7QAuB/BpVX0zgB/iZFinCWnYp6ftUD2oquuqur5r1y4vYwkh+cNXcuaPj/AfA3BMVR8w3+9A0RB8vwzhmL/PVo6/qPL7CwE8ncZcQvKDA5mnwldy5o9T+FX1ewCeEpE3ml17AXwTwF0ArjX7rgVwp/l8F4APm9k9VwJ4qQwJEbJocCDzdPhKzvwR1dOiMKcfJPImAJ8BsBPAdwD8OopG408B7AZwFMA/V9XnRUQA/CcA7wFwAsCvq+qW7fzr6+u6tWU9hJAs2bOnEPs6a2vAE08MbQ2ZGyJyWFXXQ3/n9c5dVX0QQNPJ9zYcqwB+O9QQQqYIBzLJFOGTu4R0gAOZZIpQ+AnpAAcyyRSh8BPSAQ5kkiniFeMnhLSzsUGhJ9NiITx+zqOeLsw7QoZn8sLPedTThXlHUkEHIgyvefx902UeP+dRTxfmHUlB6UBUF4ZbXp7HWEvsPP7JC/+2bYW3WEcEePXVjoaRXmHekRTM2YGIFf7Jh3o4j3q6MO9ICvgQXTiTF37Oo54uzDuSAjoQ4Uxe+DmPerow70gK6ECEM/kYPyGEbG4Wb/g6erTw9A8cmIcDMdsYPyGEbGwUA7mvvlr8zUn0c5xqyid3CSGkJ+pTTctnVYBxGyd6/IQQ0hO5vniewj8QY3X3cuxmEjIXcp1qylDPAIzV3cu1m0nIXNi9u/nhsrGnmtLjH4Cxunu5djMJmQu5TjWl8A9Aqu5eaNgm124mIXMh12dVKPwDkOLJwpiVLLtcl2MD+cM8SkefaZnlVFNVHX274oordJE5dEh1eVm1kOxiW14u9vuytnbq78ttbS39dVPYO0cOHSryQ6T422d6MY/SMeW0BLClEZo7uujrDIRftbsoiDQLv0j668Y0MnNnaPFgHqVjymkZK/xcsmEiDLn07NyWS07xuP/QSwPPLY/6ZMppySUbFpwhZwfMabXDVG8BG3ogfU551DdzTEsK/0QYcnZArlPQ+iDVlNehxWNOedQ3s0zLmPhQ6m0OMf6pMeRA5ZjEjp3UGWOAcC55NARTTUswxk9IOClj83NdGpiMB2P8hESQspuf5XxtQhqg8JNZk+uTlcQOH17rBhdpI7NnY4NCPyW4+GB36PETQiYFFx/sDoWfEDIpuPhgdyj8hJBJMccHrlJD4SeETIpZPnCVGAo/IR5wFkk+cCZWdzirhxAHnEWSH5yJ1Q16/IQ44CwSsmh4C7+ILInI10TkbvP9YhF5QEQeF5HbRWSn2X+G+X7E/H9PP6aTXFj0MAhnkZBFI8Tjvx7Ao5XvHwHwcVW9FMALAK4z+68D8IKq/gyAj5vjyIKSalnjnOEsErJoeAm/iFwI4FcAfMZ8FwDvBHCHOeQ2AB8wn68232H+v9ccTxaQOYRBOIuELFqv1tfjvwXA7wEo30ezCuBFVX3FfD8G4ALz+QIATwGA+f9L5vhTEJF9IrIlIlvHjx+PNJ+MzRzCIJxFMl82N4HzzgN+9VcXq1frFH4ReR+AZ1X1cHV3w6Hq8b+TO1QPquq6qq7v2rXLy1iSH3MJg3DlzflRhjGfe+70/029V+vj8b8NwPtF5AkAX0QR4rkFwNkiUk4HvRDA0+bzMQAXAYD5/1kAnk9oM8kIhkHIotIUxqwy5V6tU/hV9SZVvVBV9wC4BsB9qroB4H4AHzSHXQvgTvP5LvMd5v/3aQ5veyG90BQGufbaotIsSjyUzBOXsE+5V9tlHv/vA7hBRI6giOHfavbfCmDV7L8BwI3dTCQ5YBvcqoZBDhwAbrttseKhZJ7YhH3qvVq+epE4qT+5ChQFv2mAM+WrDAkZk6ZyDwCrq8AnPpHHOA9fvUh6I2TK5hxm+ZB50BTGPHQI+MEP8hD9LlD4iZMQMe86y2fR5kuTabOos7ko/MRJiJh3meUzh6eACckBCj9xEiLmXR52msNTwITkAAd3iRebm4UAHz1aePoHDqTv9m7bVnj6dUSKrjYh5FQ4uJsIxpibGSLWOZengAkZGwp/BcaYx4VPARMyDBT+CowxjwsXQyM5sci9fwp/Bc5BH59FnT6XkhwFKUeburDovX8Kf4VcYsyLVolyoy19p5DuXQWpj3tcRJFc+N6/qo6+XXHFFZoDhw6pLi+rFsW32JaXi/1zsmGRaUvf/funke5ra6faWG5ra+7f9lW2uthU2rW2pipS/M0hzUWa70lkbMtOBcCWRmju6KKvGQm/6viFsGsl8mHsexyTtvRdWuo/3VPQRZD6KltdbMrV0RmiHqaAwr8g9O1p5FrRhqItfds233QfqjHtIkh9la0uNuUqsFOpJxT+BaHvipBrRRuKPjz+IUWiy7X6yvsuNuUcUplCz5jCn5AxM7xvEcm5og1BHzH+oRvT2PLZZ9mKtWnujkhXKPwJOHRIdXX19EI4xgBvXw0PK1p7+sam+5Qa09y82KmEVHKFwt+RpgK4iMLIinaSVCLIxjSOMv2robYcGqMpQeHvSFvlzdl7iyU3r28MUjaAbEzDmVKa5VxfKPwdcc32yNF7y7lA5o6vl+6bxsyLMKbSS8q9gaLwd8Tm8Y+V0TYxSVUg5ypYPnH53Cv9lJnKuEjuDRSFvyNtMf7V1fFE3yY6KQrkWMKWQ2Pjk36L+ERqLuQuqCW5N1AU/gTkVFFdFSNFgRyj8uXiRfvYYQv/ucpHLvfZBynqSVP6iBTTanMi9waKwj8AQzYMLmFPUSBtwtbXPeZUkVz56Rrwtwl5TveZkpQN2v79p5fB3BrH3BtwCn/PDF0AXMKRwh6XsPVxj7l3nau4pvjahHyo+xy6l5qyQRurxxmaXjlFAupQ+HtmjKczXcLetUD6CFvqe5yaJ1ydax4i5EPc5xjeaMoGbWgnIHfvPQYKf8+M4akO4WlUrxEqbjH25rhsgA+hQj6EyIzRiE7Z45+a0+EDhb9nFrHQ1OlyjyFC14dA9y20Mefvu+EeyxmZ6oNvUwoz+kLh75lF7CbW6XKPYzeMIdePFeTcYr1N60oNkeYp02HINB27jPYBhd+DFDHxnCp+H8Te49jelO/1F6UBP3RIdceO0+93587p3ctQLEreV6HwO2jK9B07Cq9pkYV8KMb2pnyvP7adqWi7j9XVsS3Lj6ozs7q6WHU+VvgX9mXr9ZdKX3/96S9Pfvll4LnniiqzCC+IHpMDB4Dl5VP3LS8X+3O6/tGjzb9v2z80vi9Db7P3+ef7smya1F8E/9xzwN//PfD5zwNPPAFsbIxt4UjEtBapt9Qef9ODIb7b1Dy/nBgqFNZ2HZ/r5+zxh4Qicr6PJoYMk1avNZV3KccChnoKDh2KF/0UMekuBXwOYwhd6RqnzTnOGzpA3fd9pCqPQ6a577MpU57JU4XCb/B5GrUvj79LAc9ZkHIihaebawMbOkDe532kLI9D9k586z89/gUTfpu3v7p66iDPzp1phbat0C0tuc87ta57KkLFa+zZQ32SUxlIacuQeebT218kh4rCb2grsCKnZ3Zqj8lW6FyFbZEFrY2YmVY5iWNqcun1HTpkF86cX6huc75y6+GlgMJvaKo8Qy336upm2gr6UJUjpzCHT7e8aX2iHMTRh5C0rq4JlPr9syF2hEyMyDGMOaXykQIKfwXfNWOGWDbA13t3FdgU9uZWKXwFphqiW1srxCmXxquN0CUs+ly/KMSO0IkRvo7JWLN6hqz/Y9Cb8AO4CMD9AB4F8AiA683+cwHcA+Bx8/ccs18AfBLAEQAPAbjcdY2hl2zou6LFTiGzTVNMYW9uYZLYgfgpeHAhad1nvqSww7blFIr0FXNbfXKdI6YX12fj0qfwn1+KN4DXAfg2gMsAfBTAjWb/jQA+Yj6/F8BfmAbgSgAPuK4xtPD3LYCpG5ZU9uY2juA79W7sxiqmAoekdZ/5ksKOXPLBRopnIFZX3b3uHHpxVQYL9QC4E8C7ATwG4Hw92Tg8Zj7/EYAPVY7/yXFt25DCbxu4SimAKVv7VMKQeipkisffq+dbWfEXnKEaq9gK3IfHHxPCCFnILdTjz2k57ZD0jg1n5dKLqzKI8APYA+AogNcDeLH2vxfM37sBvL2y/14A6w3n2gdgC8DW7t2706ZGCy4PMxfvpU6qQtQ28F2ey9Wt3b/fnn5dhKDNtjPPHDevYtM+tXcYe8yOHfZpy/WGvH5sfUs9OyaVZxzyGtHQBq50MnLpxVXpXfgBnAngMIB/Zr63Cf+fNQj/FbZzD+Xx2zI857hxym5jdfZIvXC6urU+nlKsILd5piLpn7fwoZpOsRU4ZTzYpwGyhTB8x462beunjrTdn0+PxCcdQ2aItdUnmy2hY3cL4fED2AHgywBuqOybXKjHJly5in5J6oEiV8GMHXiN8Whs4bdSjEpBWlrqf2quz9hDyGyWqqCsrsblnY8HGeplhuSxz0OIbbQJ7f797nLk6/T4jheV+VbPF6AINTY5Ga6eblOedhlADqHPwV0B8DkAt9T2/4fa4O5HzedfqQ3ufsV1jdTz+NsStc9WOGRGQQ7TyFxd4xjRj03LsWLLsfY0hVjaPOpUa+Z38fjb8iQkn7uEKNrsavOgqzaH3FM1H3walKa8WVo6fdwqpDfRZk95rtSDvn0K/9sBqJma+aDZ3gtg1YRxHjd/zzXHC4BPAfgbAN9oiu/Xt1TC70rUvkbau3glY4WYunj0tv/HeOMpZ5OkaFht9tTP2RZXbwsbuOxvIzbGbxvDCSkDXZyjmPwtbY2NlXfp0dbv1dd+nzRK7XwONqunj62L8Fcruk8Mrg+P2zczbccN3ROImUpZdntDKo0PMY1QuQSHa/DZ1zv3sadplo3Na3XZH0rIrJ6q6JdbfTmMpvRqGgwG2kNUXdKzLe2qL5NJObhefYLfp1fgssFWl9tIPeg7S+H3Fa++p/75ZqZrLR+bYPWBT9e4vI96gU5ZgGMaoaY51202VeO6qXpmMTaHCFdXfMMTTU9BN8W/fdMgJD19G+rYutG0/ET52xDn5dChsF6LzT56/JUtVvh9W+KxKpmvx9/lZREhXmAKLzf2+JD7qD4f0LaKqiuUUm8QfGxus6FrmKS+DfFe3K7hiVRjClWRbVqDaP/+k/uXllT37m1uiNrGUGzl2tXD9hl/8Y3xN9Xptro4iRj/EFus8PsU7qE8Z58R/DYRcwlW9ToxoY0+xh+6FuCQ0FbTsSEeWClEtl5K6P3EDoDHzuoJxVes2sqYz/GuXl/MmJtv/bV58yH2VR2IlRV3KKw8Z+kYxNg+iVk9Q2ypPf4xlmD1HcFvWnbY13MKDW3Y0sg1K8In7WI9sRCRbfMYfT3+6nltaZGqxxMiAH3iG4pylbEuZavLAKstD2wNf5eyn7J+lVt1vKIPZin8vrHYIQdNq/gWPB/vJURouszt7opPnoQMnrYJUVM3vX5/9fy22RaaTr7CWvcihxy38QmZhZSx0N6kK019e00hg63VY0McjNj65VMO+szzWQq/ql3YXSP7IeeNWQrYVrBtNlaF0FWJXAKaOhbvwud6viIbG0e33VtbzyxmnMVVRnxCcS5CQ2Ku0IrtXCHTWH3sa8u/0guO9fhtdtY9bN/0i61f5TVss7u6PPzmYrbCb6OtYJUx3Ta6xB59rg+cbHx844S+lSTlrIgYfETdJQil3TGiHyOsbXndNZ26Nrqhedf1en0M2NsGUGPrmasuxPSsbFrhk/6u8tpXnaPwN+DyYNqI9UTquEIItkfWmwpyWyWxPaRT2jFUuMt3vKKrIFS3LmM6tnGirunUNcwWKsRdr9eHk+Bai8clmDEiC7hf4el77yE9/dQP7flA4W/ANxZYJzb22IRLsEKuY+tSlgtxVRuCcn/ITJuu68o0ifqOHaefxyYIoSGeLuMVfY6BdPWgQ20LGTuxhUdDQku2WTE2x6e0SyQuzBYynbcU8djJCr7HuRyWPsbVKPwN+I7+1wnx+F2FJTZO3WZrzDRCn3njLi88JM3rg4hN57BVjtB77OJJ9TkG0tWDttnWNL6Q8qErn3tzDbDbnrdw5bHLJt/esk+ehjg8PuMoXZ7LCYXC34LPjJk6Lk+lPIfvPPq2c/l4/PXuZuwSAa5CZ2ugQgqsb6inS5pUt9AufZ2+x0Bs3qHLu7SFH3xCYU0Clqqh83Vomp6wbttCQnYxDlWTxx3q8PiMBbSt8skYf23rU/hjwxcuEbUNuNYrUVvj07ZWiushktjNVqlcDZ0vPg/NxDZedYHwmaLoQ9cufuhxIY1N07m6jEHFhrbqdoSUuXod9BHmLjOQQtIj1OGJuW6fD+1R+BtoqmA+0zl9u2s+0zWr54zx/FKEiupb6EyJpSX/NHeFJ7o2Yr4PZKWgmje+jYyPqHexO2S2U5OYx1y7rR75iq1vvlcHfGPT0JYWoQ5PSPq5ymzTVN8UEy4o/A3YumUhMbzQwueaLhpCjIfhWyHr92073hdbpe3aiNU9J9fAIdC8Pkzsffiko4+wusSmtLUpjh/ScDblcZvnXb7opimdQgdSq5tvvvs26NXzNaVjU5il7uxV09XnnQD1NHSNbcRssaEgCn8DMdM5Q6b3xQ4ehxAilmUF9WksmryZkBdz22jrxbgEL7Ry+MRbYyuXb7rX07HLcwzVrWl55BAnwGdQ1+e3LofA1iiUz2W47K7XLdvxPvdgG/fxTYe2GH/T5IVUW4xmUPhruGLJbTFNn+ln1cIUev6qffVK4/sKt5CCY2vIfKajpRyUstkS4s1WPeLQEIRv5QoJZfjcY31wO9W4Tbm5Qgehva3SXp8YuKvc2M4REnYMGRuq2lZNF59ZRqur7enZR+i1akMosxR+W9w8ZLGpKr5epCt84YqZ2rqL9aVpq4VwZcWv4Li6xG2Vz3cAMwafqXDltX0qyf794YOObZXLVyBcouXbeKYIffmUtZLQkGGZTrbf1T3ptnLTVh9tL3dpSsNQ+0MbWFtIzbWm01D5WGd2wm8rTLHLpbad1xbOifGSQyt8eT5XQbZ5YKkLXWwD4fs7XxEP9Wzb4rb17vvS0un7qiEE23r9IbN/usaLXWXZJ5ZtS6e29FxZCcv/0PJSH1gPHWco350bk6a2+t7X5AXG+D2J9ZZiCqntfE3Hu84fOxXNt8scmzYhlbLv9X9iB1dDBudL2gRi587TxSdmGmlb+eji8fs0KilEqOk8TeMPfc1T7yMs1iXNYycvNE3T5qyeCOFPNY+3xCbebd5SyDTHKjEV3vVEq+/gWGjFD7E/1YB2iU84pO2hnKYwVz204HP+1dUi9OaTpm29idiwQZfBdpe3K6J62WWnl+0mEfINg6XOf9V+Yurl8iah9aRp3KDqENh+20ejqKo6O+EPLRC2pQdcHqztvG3ns7XmMV18m8dfLci+8ekYESuJfQioDVd62e7H1Zi35WsfnmT1ITWfBbtcA6dt3rbtSWWfa1ftbSvztjxx3X9KUsfUY2eG+Y7nhJbRrsxO+EMrru1NOC4P1iW4wEnPKeRBn9C3SPl2u5v2uQp2SCVO6fG3jalU51379nTq57U9hNeHJ1kKpE+jXoYNbMfWQwKuspWiMXONW9l6KmN4/Csr9vJRd4p8B+N9QjE+ZajrkiIuZif8quHeTf23ru5e1YPzEVxXhbLdR9ssHtfAYdv91wt86PS0WE/adl8hlac6cGtreNvSzyWAfXmSvg1KmR4hExF8HJCum21cyzZgmvKhxXoZasvLnTvt6Vd3IGzXiIm3u7Qj1ZIiNmYp/CU+sxeqQhbTRfMVXFtB6HJfbQWya9gldqDWJ5zlc17XcxNt5yoFz/ZC7LZtaSmNUDZ5cr4PooU+t+B77q5bl3WUUgxWtpW1pvrm07Pqa9BZNT5SkLJnNGvhr+IjOD6V3lVgQith6EuXfYXTp3D5iHTqSutb6G15UX8uoanyx4qhz+Cqza66Jxni7YdOTyzToY/wVNv1uv4mteDaeiEu2/qKsbvqaOqxsCYo/BXahMx3loiP+IVWwlDh9xVOmzfcNi7gWylDGoT6sT7pERozTiV85VPLMb/17d2k3FZWhrlO6q2L4IaUJ5+wXl/Y6sgQs58o/A58Kk1IhoRWQtsYQ1lgfGK+ZSy1HtNuewlHbOFzNSiuY32EtZxptXdvs+0hYaEhtrbG2yVM5QJoXbayh1F1XkLe4FbmeYoZX64B1bYy70tIeSrrji0PXAO8fdA2cB/6YiMXFH4HPl5DaIaEVELXGMOOHX4VuenFFjaBd9nUds+u9Ko2AL5T4nzvp21QLtbjL9O163sAgLhxlhQNVnXw1OV0pFikzFZPfMtarGfrW57q5cR3Eka1IRw6/h/a83dB4XfgGkTsWgBsU/PqrXysgMUKvEtQmtLAd5DSFSuPvdcmL60tbHXmmfbrl/njuqZPA9EkZrGDfKGbz/l8ynK1x5i6PNXLhq8dvgPkrrGE0EkYfQ3+DhHfV1Wl8DvoMsLeZVpi9TrllMrYSh8zICni7+HFLPuwtjaM8FXnkdfzwjXP3+XlNo2b2NKzqXy4Fp/zyQPfZytSi0ofzoTPy45Clz6IeZdt7BPXXRliRo+qKoXfgc8gZ1vcveu0xK5btbDEiGjo/PK29GogSE0KAAAIqklEQVSr5D7Clzodqri61bZ7D12/p80G35lTrnuzvUTcdT+xotLHoLHLFts9tJUnWxkMvY7v72PpMqkiBAq/B7bK2ZZRvgLQ1atti/HXw0Rd7AytBDYRqp/fJXyuNz/5bPXpndVwhe1Bmdgnf/uouD7nbUv3siz0YVvThIHye0x5dompq9fSVJ5iGrzUkzpC6GOadB0Kf0dChbtplk6s19Q2q8e2XnlMzyS0EvikiW9hDvXiXI1MyBo2sR5yXxXX57yuhnwIUSmxpV9I2lZtjgnbxDZ41Uaj7+cNhobC35FQzya2YPfpbdi8tmoD4VsJXGkS81Carxdnsy1UyIfqdqdkqMFBH2zp55u2Pk5HiifGfe4l5PdDNrAxUPg70uZhrazEexk+D5b0VZBixzSqxMTGU9lpW68oRhRzr8B1hhoc9KWtl1na6XqpvW3A1uaojMkQIbWu90jh74htsbPYzKqHbuqbzwJSsaQQjrbGqy0EFUto+uYmirHEjDmNLYYlofb5xPRzu98hBtG73iOFvyN9da1dM076ItX95Ogp5ygSoaTokY1JqCi6js+xMU+tCX3cI4W/I30VvLFitTlWpJTkLIo+TD1/Qsu1q6HLaUyjJHUe9XGPWQk/gPcAeAzAEQA3uo6PfRHLykpzQnKb3ra0VKzZ45pJ1OfzEqUNfT3NOsWtaaps6k1Edfv2+N93+W0uW2z4NFb4xQh1MkRkCcC3AbwbwDEAfw3gQ6r6zbbfrK+v69bWlvc1NjeBD38YePXVrtYSQkge7NwJfPazwMaG/29E5LCqrodea1voDzz4BQBHVPU7qvojAF8EcHXKC9x8M0WfELJY/OhHhbYNQR/CfwGApyrfj5l9pyAi+0RkS0S2jh8/HnSBo0e7GUgIITkylLb1IfzSsO+0eJKqHlTVdVVd37VrV9AFdu+ONY0QQvJlKG3rQ/iPAbio8v1CAE+nvMCBA8C2PiwnhJCR2Lmz0LYh6EM+/xrApSJysYjsBHANgLtSXmBjA/jc54CVlZRnHYalpbEtyJOlJWDvXmBtzX6cNPUnE9uwutrfNdo444zhr+nD6mohSH0iAlx2WXzebt+e1p4xWF0NH9jtQvIkU9VXRORfAfgygCUAn1XVR1JfZ2NjuEQihJBFope2UlX/HMCf93FuQggh3WCknBBCZgaFnxBCZgaFnxBCZgaFnxBCZkbytXqijBA5DuDJyJ+fB+AHCc1JDe2LJ2fbANrXhZxtA6Zj35qqhj0Bi0yEvwsishWzSNFQ0L54crYNoH1dyNk2YPHtY6iHEEJmBoWfEEJmxiII/8GxDXBA++LJ2TaA9nUhZ9uABbdv8jF+QgghYSyCx08IISQACj8hhMyMSQu/iLxHRB4TkSMicuNINnxWRJ4VkYcr+84VkXtE5HHz9xyzX0Tkk8beh0Tk8p5tu0hE7heRR0XkERG5PjP7XiMiXxGRrxv7/sDsv1hEHjD23W6W94aInGG+HzH/39OnfeaaSyLyNRG5O0PbnhCRb4jIgyKyZfZlkbfmmmeLyB0i8i1TBt+Sg30i8kaTZuX2dyLyOznYVrHxd02deFhEvmDqSrqyF/OG9hw2FEs+/w2ASwDsBPB1AJeNYMc7AFwO4OHKvo8CuNF8vhHAR8zn9wL4CxRvKbsSwAM923Y+gMvN59cB+DaAyzKyTwCcaT7vAPCAue6fArjG7P9DAPvN598C8Ifm8zUAbh8gf28A8J8B3G2+52TbEwDOq+3LIm/NNW8D8Bvm804AZ+dkn7nuEoDvAVjLxTYUr6r9LoDXVsrcr6Use70nbI+J8xYAX658vwnATSPZsgenCv9jAM43n88H8Jj5/EcAPtR03EB23gng3TnaB2AZwFcB/CKKJxK31/MZxTse3mI+bzfHSY82XQjgXgDvBHC3qfhZ2Gau8wROF/4s8hbA6414SY72Va7zSwD+V0624eR7y881ZeluAL+csuxNOdTj9VL3kXiDqj4DAObvT5n9o9lsun9vRuFVZ2OfCaU8COBZAPeg6MW9qKqvNNjwE/vM/18C0Of7sm4B8HsAXjXfVzOyDSjeZf2XInJYRPaZfbnk7SUAjgP4ExMq+4yIrGRkX8k1AL5gPmdhm6r+LYCPATgK4BkUZekwEpa9KQu/10vdM2MUm0XkTAD/FcDvqOrf2Q5t2Nerfar6Y1V9Ewrv+hcA/EOLDYPZJyLvA/Csqh6u7rZcf4y8fZuqXg7gKgC/LSLvsBw7tH3bUYRAP62qbwbwQxThkzYGTz8TI38/gP/iOrRhX2+2mbGFqwFcDOCnAaygyOM2G4Ltm7Lw9/5S9w58X0TOBwDz91mzf3CbRWQHCtHfVNUv5WZfiaq+COCvUMRQzxaR8u1wVRt+Yp/5/1kAnu/JpLcBeL+IPAHgiyjCPbdkYhsAQFWfNn+fBfDfUDScueTtMQDHVPUB8/0OFA1BLvYBhZh+VVW/b77nYtu7AHxXVY+r6ssAvgTgrUhY9qYs/L2/1L0DdwG41ny+FkVsvdz/YTNL4EoAL5Vdyz4QEQFwK4BHVfU/ZmjfLhE523x+LYoC/yiA+wF8sMW+0u4PArhPTWAzNap6k6peqKp7UJSt+1R1IwfbAEBEVkTkdeVnFLHqh5FJ3qrq9wA8JSJvNLv2AvhmLvYZPoSTYZ7ShhxsOwrgShFZNnW4TLt0Za/vwZM+NxSj7d9GERe+eSQbvoAiDvcyipb3OhTxtXsBPG7+nmuOFQCfMvZ+A8B6z7a9HUWX7yEAD5rtvRnZ948AfM3Y9zCAf2v2XwLgKwCOoOiGn2H2v8Z8P2L+f8lAefxPcXJWTxa2GTu+brZHyvKfS96aa74JwJbJ3/8O4Jxc7EMxmeA5AGdV9mVhm7nmHwD4lqkXnwdwRsqyxyUbCCFkZkw51EMIISQCCj8hhMwMCj8hhMwMCj8hhMwMCj8hhMwMCj8hhMwMCj8hhMyM/w9xEtzpPMbB5wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x23ebcf36908>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 单个特征散点图\n",
    "fig = pyplot.figure()\n",
    "pyplot.scatter(range(train.shape[0]), train[\"Insulin\"].values,color='blue')\n",
    "pyplot.title(\"Insulin\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "846\n"
     ]
    }
   ],
   "source": [
    "#发现散点图上的一个孤立点\n",
    "#找到改点的数值查看\n",
    "Insulin_max = np.max(train.Insulin)\n",
    "print(Insulin_max)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "看一下刚才统计特性中存在疑问的Insulin特征的散点图，发现数据大多集中在400以下，但是有个别数值很高的点，所以造成了平均数和中位数的不一致。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = train[train.Insulin<600]"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "考虑可能是测量过程中存在误差，所以去除掉数据较高的大于600离群点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(764, 9)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看一下是否去除成功\n",
    "train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Target 分布，看看各类样本分布是否均衡\n",
    "#seaborn.countplot(x=None, y=None, hue=None, data=None, order=None, hue_order=None, orient=None, \n",
    "#                  color=None, palette=None, saturation=0.75, ax=None, **kwargs)\n",
    "#sns.countplot(train.BloodPressure);\n",
    "#pyplot.xlabel('BloodPressure');\n",
    "#pyplot.ylabel('Number of occurrences');"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "该方法对于该数据的分析没有意义，经过测试该方法可用，直接注释掉"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 特征编码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将类别字符串变成数字\n",
    "# drop ids and get labels\n",
    "y = train['Outcome']   #形式为Class_x\n",
    "#y_train = y_train.map(lambda s: s[6:])\n",
    "#y_train = y_train.map(lambda s: int(s)-1)\n",
    "\n",
    "train = train.drop([\"Outcome\"], axis=1)\n",
    "X = np.array(train)\n",
    "\n",
    "#如果计算资源有限，也可只取少量样本，如取前1000个样本\n",
    "#（分类中其实还需要确保取出来的这部分样本各类样本的比例和总体一致）\n",
    "#n_trains = 1000\n",
    "#y_train = train.label.values[:n_trains]\n",
    "\n",
    "#数据集样本数较少，可以直接进行后续步骤"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(611, 8)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#将数据分割训练数据与测试数据(随机选择%20的数据作为测试集)\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 随机采样20%的数据构建测试样本，其余作为训练样本\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=33, test_size=0.2)\n",
    "\n",
    "#查看一下训练集数目\n",
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(153, 8)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看一下测试集数目\n",
    "X_test.shape\n",
    "#确定划分正确"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据标准化\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# 初始化特征的标准化器\n",
    "ss_X = StandardScaler()\n",
    "\n",
    "# 分别对训练和测试数据的特征进行标准化处理\n",
    "X_train = ss_X.fit_transform(X_train)\n",
    "#X_test = ss_X.transform(X_test)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "做一下数据标准化，反正做一下总是好的"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 数据训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "#default Logistic Regression\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "lr= LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of each fold is:  [0.73170732 0.78817734 0.75369458]\n",
      "cv logloss is: 0.7578597460851455\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "# 交叉验证用于评估模型性能和进行参数调优（模型选择）\n",
    "#分类任务中交叉验证缺省是采用StratifiedKFold\n",
    "#sklearn.cross_validation.cross_val_score(estimator, X, y=None, scoring=None, cv=None, n_jobs=1, \n",
    "#                                         verbose=0, fit_params=None, pre_dispatch=‘2*n_jobs’)\n",
    "\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "#loss = cross_val_score(lr, X_train, y_train, cv=None, scoring='neg_log_loss')\n",
    "#logloss of each fold is:  [-0.52290392 -0.48305742 -0.50267186]\n",
    "#cv logloss is: -0.5028777346865436\n",
    "\n",
    "#loss = cross_val_score(lr, X_train, y_train, cv=None, scoring='f1')\n",
    "#logloss of each fold is:  [0.58015267 0.67669173 0.57627119]\n",
    "#cv logloss is: 0.6110385291732371\n",
    "\n",
    "loss = cross_val_score(lr, X_train, y_train, cv=None, scoring='f1_micro')\n",
    "#logloss of each fold is:  [0.73170732 0.78817734 0.75369458]\n",
    "#cv logloss is: 0.7578597460851455\n",
    "    \n",
    "#loss = cross_val_score(lr, X_train, y_train, cv=None, scoring='f1_macro')\n",
    "#logloss of each fold is:  [0.69151003 0.75959129 0.70133004]\n",
    "#cv logloss is: 0.7174771170686647\n",
    "\n",
    "#loss = cross_val_score(lr, X_train, y_train, cv=None, scoring='f1_weighted')\n",
    "#logloss of each fold is:  [0.72355922 0.78368525 0.73767719]\n",
    "#cv logloss is: 0.7483072183445821\n",
    "\n",
    "#loss = cross_val_score(lr, X_train, y_train, cv=None, scoring='average_precision')\n",
    "#logloss of each fold is:  [0.65385864 0.70982668 0.73383177]\n",
    "#cv logloss is: 0.6991723627532828\n",
    "print ('logloss of each fold is: ',loss)\n",
    "print ('cv logloss is:', loss.mean())\n",
    "#neg本身为负值，所以在打印结果时前加一个“-”，即输出为正值"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "选取了不同的scoring，因为scoring越大的模型性能越好，所以最终选择fi_micro模型\n",
    "此处还尝试了不同CV的交叉验证，其中默认的3折验证表现的性能较好"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 正则化的 Logistic Regression及参数调优"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "logistic回归的需要调整超参数有：C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1） \n",
    "目标函数为：J = sum(logloss(f(xi), yi)) + C* penalty\n",
    "\n",
    "在sklearn框架下，不同学习器的参数调整步骤相同： \n",
    "设置候选参数集合 \n",
    "调用GridSearchCV \n",
    "调用fit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=None, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'penalty': ['l1', 'l2'], 'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='f1_micro', verbose=0)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "#需要调优的参数\n",
    "#tuned_parameters = {'penalty':['l1','l2'],\n",
    "#                   'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "#                   }\n",
    "penaltys = ['l1','l2']\n",
    "Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "tuned_parameters = dict(penalty = penaltys, C = Cs)\n",
    "\n",
    "lr_penalty= LogisticRegression()\n",
    "#sklearn.model_selection.GridSearchCV(estimator, param_grid, scoring=None, fit_params=None, n_jobs=1, iid=True, refit=True, cv=None, \n",
    "#                                     verbose=0, pre_dispatch=‘2*n_jobs’, error_score=’raise’, return_train_score=’warn’)\n",
    "grid= GridSearchCV(lr_penalty, tuned_parameters,cv=None, scoring='f1_micro')\n",
    "grid.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([0.00133387, 0.00200184, 0.00266902, 0.00133491, 0.00133499,\n",
       "        0.00200137, 0.00200248, 0.00233523, 0.00300209, 0.00166758,\n",
       "        0.00133546, 0.00200232, 0.00233515, 0.00133411]),\n",
       " 'mean_score_time': array([0.00133475, 0.00100017, 0.00166901, 0.00367053, 0.00133348,\n",
       "        0.00033339, 0.00133316, 0.0006667 , 0.00200415, 0.0010008 ,\n",
       "        0.00099929, 0.00133268, 0.00100009, 0.00066741]),\n",
       " 'mean_test_score': array([0.64484452, 0.73322422, 0.64484452, 0.74304419, 0.74959083,\n",
       "        0.7512275 , 0.7594108 , 0.75777414, 0.75777414, 0.75777414,\n",
       "        0.75777414, 0.75777414, 0.75777414, 0.75777414]),\n",
       " 'mean_train_score': array([0.64484529, 0.74551257, 0.64484529, 0.75203242, 0.75694646,\n",
       "        0.76432355, 0.76350655, 0.76677453, 0.76677453, 0.76677453,\n",
       "        0.76677453, 0.76677453, 0.76677453, 0.76677453]),\n",
       " 'param_C': masked_array(data=[0.001, 0.001, 0.01, 0.01, 0.1, 0.1, 1, 1, 10, 10, 100,\n",
       "                    100, 1000, 1000],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'param_penalty': masked_array(data=['l1', 'l2', 'l1', 'l2', 'l1', 'l2', 'l1', 'l2', 'l1',\n",
       "                    'l2', 'l1', 'l2', 'l1', 'l2'],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'params': [{'C': 0.001, 'penalty': 'l1'},\n",
       "  {'C': 0.001, 'penalty': 'l2'},\n",
       "  {'C': 0.01, 'penalty': 'l1'},\n",
       "  {'C': 0.01, 'penalty': 'l2'},\n",
       "  {'C': 0.1, 'penalty': 'l1'},\n",
       "  {'C': 0.1, 'penalty': 'l2'},\n",
       "  {'C': 1, 'penalty': 'l1'},\n",
       "  {'C': 1, 'penalty': 'l2'},\n",
       "  {'C': 10, 'penalty': 'l1'},\n",
       "  {'C': 10, 'penalty': 'l2'},\n",
       "  {'C': 100, 'penalty': 'l1'},\n",
       "  {'C': 100, 'penalty': 'l2'},\n",
       "  {'C': 1000, 'penalty': 'l1'},\n",
       "  {'C': 1000, 'penalty': 'l2'}],\n",
       " 'rank_test_score': array([13, 12, 13, 11, 10,  9,  1,  2,  2,  2,  2,  2,  2,  2]),\n",
       " 'split0_test_score': array([0.64390244, 0.72195122, 0.64390244, 0.72195122, 0.73170732,\n",
       "        0.72682927, 0.73170732, 0.73170732, 0.73170732, 0.73170732,\n",
       "        0.73170732, 0.73170732, 0.73170732, 0.73170732]),\n",
       " 'split0_train_score': array([0.6453202 , 0.75369458, 0.6453202 , 0.74384236, 0.75123153,\n",
       "        0.76600985, 0.76600985, 0.76600985, 0.76600985, 0.76600985,\n",
       "        0.76600985, 0.76600985, 0.76600985, 0.76600985]),\n",
       " 'split1_test_score': array([0.6453202 , 0.74384236, 0.6453202 , 0.75369458, 0.76847291,\n",
       "        0.78325123, 0.78817734, 0.78817734, 0.78817734, 0.78817734,\n",
       "        0.78817734, 0.78817734, 0.78817734, 0.78817734]),\n",
       " 'split1_train_score': array([0.64460784, 0.73039216, 0.64460784, 0.75      , 0.76470588,\n",
       "        0.7622549 , 0.7622549 , 0.76715686, 0.76715686, 0.76715686,\n",
       "        0.76715686, 0.76715686, 0.76715686, 0.76715686]),\n",
       " 'split2_test_score': array([0.6453202 , 0.73399015, 0.6453202 , 0.75369458, 0.74876847,\n",
       "        0.74384236, 0.75862069, 0.75369458, 0.75369458, 0.75369458,\n",
       "        0.75369458, 0.75369458, 0.75369458, 0.75369458]),\n",
       " 'split2_train_score': array([0.64460784, 0.75245098, 0.64460784, 0.7622549 , 0.75490196,\n",
       "        0.76470588, 0.7622549 , 0.76715686, 0.76715686, 0.76715686,\n",
       "        0.76715686, 0.76715686, 0.76715686, 0.76715686]),\n",
       " 'std_fit_time': array([4.71595177e-04, 8.17508124e-04, 9.43639921e-04, 4.72887650e-04,\n",
       "        4.73337217e-04, 8.77806426e-07, 7.78671819e-07, 4.71258002e-04,\n",
       "        2.16200335e-03, 4.71595137e-04, 4.73337297e-04, 1.07214749e-06,\n",
       "        4.72325773e-04, 4.72269487e-04]),\n",
       " 'std_score_time': array([4.72157095e-04, 9.73339773e-07, 9.44538993e-04, 1.24797903e-03,\n",
       "        4.71033219e-04, 4.71482745e-04, 4.71427042e-04, 4.71427042e-04,\n",
       "        1.41883172e-03, 7.86741172e-07, 1.58547857e-06, 4.69740686e-04,\n",
       "        4.89903609e-07, 4.71932312e-04]),\n",
       " 'std_test_score': array([0.00066942, 0.00896053, 0.00066942, 0.01498828, 0.01503278,\n",
       "        0.02363398, 0.0230791 , 0.02325113, 0.02325113, 0.02325113,\n",
       "        0.02325113, 0.02325113, 0.02325113, 0.02325113]),\n",
       " 'std_train_score': array([0.00033581, 0.0107038 , 0.00033581, 0.00765304, 0.00568768,\n",
       "        0.00155661, 0.0017701 , 0.00054071, 0.00054071, 0.00054071,\n",
       "        0.00054071, 0.00054071, 0.00054071, 0.00054071])}"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7594108019639935\n",
      "{'C': 1, 'penalty': 'l1'}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "只关注模型的性能，相较于交叉验证之前损失小幅升高。\n",
    "C=1可知模型惩罚中规中矩，l1模型更好。\n",
    "此处尝试了不同的CV值，但是并没有太大区别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl8VfWd+P/XOzf7SiALS9gJYUvYwqJBVEBFoFZ0VGzVajtD51ftdOzUb+0ybp126vTntN+Z8TdqrVXbjtSx1UJAQbSKssgmexYCsgQCCSELIeu99/37415CCAm5JLm5Wd7Px+M+7jnnfs657wPJeefz+ZzP54iqYowxxrQlKNABGGOM6RksYRhjjPGJJQxjjDE+sYRhjDHGJ5YwjDHG+MQShjHGGJ9YwjDGGOMTSxjGGGN8YgnDGGOMT4IDHUBnSkhI0BEjRgQ6DGOM6VF27NhxRlUT2yrXqxLGiBEj2L59e6DDMMaYHkVEjvpSzpqkjDHG+MQShjHGGJ9YwjDGGOMTSxjGGGN8YgnDGGOMTyxhGGOM8YklDGOMMT6xhGGMMcYnfk8YIrJQRPJEpEBEHm/h81+KyC7vK19Eypt89m8isl9EckTkP0RE/B2vMf5wz4ubuefFzYEOw5gO8etIbxFxAM8DNwGFwDYRWamqBy6UUdVHm5T/NjDVu3wtkAVkeD/+FLge+MifMRtjjGmZv6cGmQkUqOphABFZAXwZONBK+XuBJ73LCoQDoYAAIcBpv0ZrjJ8cCf1/vUt/CmgcHTXrt3cC8NlDPfs8wM6lPfzdJDUEON5kvdC77TIiMhwYCXwIoKqbgb8CRd7XWlXN8Wu0xhhjWuXvhNFSn4O2UnYZ8JaqugBEZAwwHkjBk2Tmicjcy75AZLmIbBeR7SUlJZ0UtjHGmOb8nTAKgaFN1lOAk62UXQa80WR9KbBFVatUtQp4F5jdfCdVfUlVM1U1MzGxzdl5jTHGtJO/E8Y2IFVERopIKJ6ksLJ5IRFJA+KBpreRHAOuF5FgEQnB0+FtTVLGGBMgfk0YquoEHgHW4rnYv6mq+0XkGRG5rUnRe4EVqtq0ueot4BCwF9gN7FbVVf6M1xhjTOv8/gAlVV0DrGm27Ylm60+1sJ8L+KZfgzPGGOMzG+ltjDHGJ5YwjDHG+MQShum2HnrvIR5676FAh2GM8bKEYYwxxieWMIwxxvjEEoYxxhifWMIwxhjjE0sYxhhjfGIJwxhjjE8sYRhjjPGJJQxjjDE+sYRhjDHGJ36ffNAYAyFON0FucJ8/H+hQOiSszkWYunGXFQc6lA7rV1sP0GvOxSktPa+uc8mlM4r3bJmZmbp9+/ZAh2E6yYVpQX678LcBjsRD3W7c587hqqjwvipxVZTjqqjAXVmJq9y7vdKz3d1YpgKtqwt0+KaXC0msZ8wnh9q1r4jsUNXMtspZDcP0Oe66OlzlFbgrK5pd/Cs8F/rLLv4VjUmBK/yBJZGROGJjccTF4YiLI3TECIK8yy8dXIUzSPiHGV/rwhN1Qv05qKuC+qpW3s9B/XlwtZTQBEIiISwGQqMgLIadZ49SIw6yhrZ5ben2Pj2+A4A5Q6cHOJKO+/T4Do71j+Hbfv4eSxim21r2n/s9Cwsv/6zxr/0mF/eWE0CF9y/9iwlAa2tb/9KgoMaLflBcHI5+/QgdPtyzrZ/n4h8U63m/sO6IjSUoLo6g0NBWD5v92y0APP3Q1zvyTwLOejhfAlWnL75XFXte54svLlcVQ13F5fuHALHxEJUE0UkQPR6ikz3LUUne5UTPe2QCOC69RPz4t3cC8NlDr3TsPLqBn/XCc7GEYfok59mzRFfUE1Lv5sR3/+nSv/S9F/4r/rUfEXHJX/shw4cRHheHI67fJRf/xgRwIRlERSFBnX8vyPCGKzQVuJxQfabJxf50s4t/k+RQU9byMcJivQkgGZInwuh5Fy/8jckhGaISIbj1xGbMlVjCMN2Gq6qKc++vp3LNGs5v2kR/lwuXQM3+fTi8f9WHDhuGIy62sanHEdcPR1xsk4u/t3YQFhbo0/Fwu6C0gGtqqkhwuWDdjy8mgSpvEqguBVpIfiFR3gt9EiSkwog5TS7+TRJAdBKERHT5qZm+xxKGCSh3bS1VH31E5eo1VH38MVpfT8jgwQz4+kNsX/l76kKEpWvXBjpM39RWwun9cGovnN4Lp/ZB8QFw1vLohTJbf32x+Sd+BAyd0eTCn3yxSSgqCcKiA3gyxlzOEobpclpfT9WmTZ4k8cEHuKurcSQk0O+ee4hddCsRU6YgIjzdfxMASwMc72VUofyYNzHs87yf2gvlRy+WiegPAyfBjL+F5Ek8tvEZih0hvPbN3dAFtz8a4w9+TxgishD4v4ADeFlVf97s818CN3pXI4EkVe3n/WwY8DIwFE+dfZGqHvF3zKbzqctF9bbtVK5ezbl163BVVBAUF0fs4kXELlpE5MyZiMNxyT5XbPfvKg01UJzjTQze5HB6f5MOZYEBo2HINJj2AAxM97xiBl2SGI5ufdZb3JKF6bn8mjBExAE8D9wEFALbRGSlqh64UEZVH21S/tvA1CaHeB34qaq+LyLRgNuf8ZrOparU7tlDxerVnHv3PZwlJUhkJDHz5hG7aBHRc7KQK9xZ1OXOnfY2JXmbk07vgzMHQV2ez0OiPLWG9L/xvA/MgKTxnltO23A0ZLSfgzfG//xdw5gJFKjqYQARWQF8GTjQSvl7gSe9ZScAwar6PoCqVvk5VtNJavPyqVy9mso1a2goLERCQoi6fi5xixcTff31BEVGBjZAV4MnEVxoTrrwfr7kYpm4oZA8CcZ/yVNjSJ4E8SPBD3dQGdNT+DthDAGON1kvBGa1VFBEhgMjgQ+9m8YC5SLyZ+/29cDjqhf+3GvcbzmwHGDYsGGdGrzxXf3Ro1SuWUPlmjXUHSwAh4Oo2bNJ+Na3iFkwH0dsbGACqylv1py0F4pzLw5Uc4RC4jhIvfliYkieCJH9O+XrXW5l34kKzpemo+rg5U8OExHqICo0mMhQB1FhnvfIZuthwUGINV+ZbsbfCaOln/jWbp5fBrzVJCEEA9fhaaI6BvwReBD4zSUHU30JeAk8U4N0POSe7ej9DwAw/Hev+/27Gk6fpnLNu1SuWUPt3r0AREyfTvIT/0zsLbcQPGCA32No5HZD+ZEmicGbJCqOXSwTmeBpSpq1HJLTPcsJY8ER0qmhnKqoZcPBEjbkl7Cx4Axl1Q3ADAD+ZXWOT8dwBIk3kXiTS5gnqUS1kFyiwoKJCHEQdaFMY9lgT3IKu7hPiMNqSKb9/J0wCvF0WF+QApxspewy4OFm+37epDnrHWA2zRKG6VrOsjLOrV1LZfZqqnfsAFXCJ0wg6bHHiL11ISGDB/s/iPpqT0f0qT0XE8PpfZ7pLgAkCAakem5ZzXzI09cwcJLnllU//NVe2+Bi6xdn+eRgCRvyz5B3+hwAiTFhzBuXzJzUeP5l+3cRRy0v3fostU4XNfUuahtc1Da4qWnwrNc4XdTWu6hpcFHn3V5bX+/Z3uB5VdS4qalwNu5X2+DC5fb976QQRxDhIQ4iQhyEhwQRHhpMREgQ4SFBRAQHExHa5PNQB+EhDsKDHZ4yocHU1UYi4mJN3o5O/3fsavV1nrE6veVcgoL838Xr74SxDUgVkZHACTxJ4SvNC4lIGhAPbG62b7yIJKpqCTAPsJkF25B7NheA4Z14TFdVFefWr6dytWdAHS4XoaNGkfDIw8QuWkTYyJGd+G1NqBLvdkH+uovjGk7thbOHQL2/HKExnmQw+V7vHUqTIHE8hPqvn0RVKSiu4uP8EjYcPMNnh0upc7oJdQQxbWQED05Q+vU7Q7nrCPll+fzrgQLqQ2sA+Nq6Ze37UgFCva8mfezh7TiUGzjvfV3C6X1difef9ftbHmzHN3cz3rGOveVc3NUj/P41fk0YquoUkUeAtXhuq31FVfeLyDPAdlVd6S16L7BCm0ydq6ouEfke8IF4GnN3AL/2Z7zmIs+Auo+pXL36sgF1sYsWETZunN/b2B+qPMut5yvhf+7ybOg3zNOUNOlOT2JIngT9hndJR3R5dT2fFpxhQ34Jnxw8Q1FFDRJSxpCkMqZklBMcXkRpwxH2VxWy/xRwCmJDY0nrn8adqXfy5v73gSD+bd73/R5rZ1CUeqdS5/TUduqcLuqcbp7f/gbqdnB/ercbHXPVfrf3HQDuT789wJF03O/2vkNIkP9b5P0+DkNV1wBrmm17otn6U63s+z6Q4bfgzCW0oYHzmzZRsXo1VeubDKi7+25iFy9qHFDXJeqrmVd9jh1hEUy/90+ejuiIfl3z3YDT5WZ3YTkf55/ho/wTHDiTj4QWER51mtjBJSSkFFLnPk8FUFkjDAsZxoQB47kjdSlp8Wmk9U8jOTK58d/rz/u3AjB/+PwuOwd/+M3e14EGHrvubwIdSoe9VfAGQK86F3+zkd5eXdlZ3J2oy0X19h2eAXVr13oG1MXGErPoVuIWLyZyxgwkOAA/Joc+IEyV1dFxTB+R1SVfWVhWzZoD+Xxw6HMOlObS4DiBI7yIoKgzRER7msAigiMYGT+WtPjFpPX3JIbUfqlEhgT4VmFjuoAljD5IVandu9czVuLd93AWFyMREZ4BdYsXEzUn64pTdXeJnGyqJIic0Pa00retwd1A7plDrMnfwWeF+zhyroC6oEKCgr0d5/0hMSSJiQnjmJg4jnH9x5EWn0ZKTApBYncamb7JEkYfUpufT+Vqz1iJhuPHPQPq5s4lbvEiom+4IfAD6i5wNUD+u+wIj8TVCU1gFXUV5Jflk1Oaw9aT+9l/JpfSumOoeHp41R1MZMgQUmNnMTtlEllDM0jrn0ZcWFyHv9uY3sQSRi9Xf+yYZ0Dd6tWeAXVBQZ4BdX//98TctCBwA+qu5MinUFvB1vikq9rNrW6OnztO3tk88sryyDubR05pLsU1py+WcUbjrh1Ev+AbSE8cz/xRU1g8PoPoMP/UZIzpTSxh9EIOp5vSV1+lcnWTAXXTppH8zz8mduHCrh1Q1x652RAcwe6w1p/xUN1QTX5ZPvll+eSdzSO3LJeDZQepcXpuXxWCCHYlU101CHftNCJIYVZKOjdNHMPcsYkkx1qCMOZqWcLowVQV56lT1ObmUpeXR21uHoOOVRHcoBT//FnCJown6bHvEXvrrV0zoK4zuN2QuxrGzKe+cj+KUlRV1FhjuPB+/Nxx1DtpQFRwNPEhw4l1ZnG+OJ6a88kENQxk2tAk5k5K4LrURCYNicMRZFNtGNMRljB6CHddHXUFBdTl5lGbl+t9z8NdcfG5zSFDh9IQ6uB8TBDTfv8OYaP8NKDOj84f/ZTdDWXs6hfHsRoXNQI3/+nmxs+HxgxldFwq42NupKw8gdyjMZw4E8YphKH9I7g9NZG5YxO5dvQAYsI7d8oPY/o6SxjdkLOkhNrcPOrychvf6w5/AS7PNFsSEUHY2FRib7mFsHFphI8bR9jYsTiio1m7eCZAj0kWRVVFfF78OZ8Xf86ukl3kn83FPSgJOf0pSBgOwvnBzEcIdg3h2Kk4thSc593tZTjdSmSog2tHD+CbWYnMTU1k+IBIm7DPGD+yhBFA2tBA3eEvLiaG3Fxq8/JwlZY2lgkeOJDwtDSi58/3JIa0NEKHDbvsYUM9gdPtJL8s35McinfxefHnnK72dEhHBEeQkZDB8togpoYnkXHv28z9zXc5fzadZ/+3P2XVlUAlEwfH8ndzRzE3NZHpw+MJDbZbXI3pKpYwuoirvJza3Dxqc3Mam5PqCwrQhgYAJCSE0NQxRM+dS/i4NMLSxhE+Lg1Hv64b3dzZquqr2FOyh89LPDWIvSV7qXZWA5AcmczUpKlMSZrC1KSpjI0fS/CZg7B5Nix6BA2JouLkDbhd4SyenMTcsYnMSU0gIToswGdlTN9lCaOTqctF/dFjl9UanKdONZZxJCQQnpZG1AP3N9YawkaOREJ6dpt7UVURO4t3NtYgDpYfxK1ugiSIsfFjuW30bUxNmsrUpKkMih50+QFysj3v4xaTU3QOV0McMckb+fd77ujaEzHGtMgSRge4qqq8dydd7ISuO3gQrfHc2onDQdioUUTOmHFJrSE4ISGwgXeCKzUvRQZHkpGYwTczvsmUpClkJGQQHRrd9kFzV8GQTIgdTPamXMBNePQRv56HMcZ3ljB8oKo0FBY2SQye94bCwsYyjrg4wsaNI/7uuxoTQ+iYMYGfYqOTNG9e2lOyp3HMw8CogUxLmtbYvJQan0pw0FX+aJUfg6LdsOApVJXsPUWERp4kKLiu80/GGNMuljCacdfUUJef3+wupTzc571PDxAhdPhwwidNot/f3ElYWhrh48cTnJzca+7QUVWKzje5e6lZ81JafBq3j7m9sXlpYNTAjn9p7mrP+7gvsfdEBcfOVhOb/EXHj2uM6TSWMPA8j7quoAB3TQ1506aD97EcQVFRhKWlEffl2xprDWGpqd1nzqVO4nQ7ySvLa2xa+rz4c4qri4FLm5emJk0lIzGDqJCoNo7YDjnZngcfJYxh9ZocgoOEsJijnf89xph2s4QBSGgo7upqgiIiiP/GNxrHNoQMGYJ0wcN5utq5+nOe5iVv7WHPmUubl6YnTe9Y89LVOn8Gjm2C6/6psTnqutQEDlDv3+81xlwVSxhAyKBBRGR4ntOU+O1HAhxN51JVTp4/eUnn9MGygyjqv+alq5X3rueRq+OW8Pnxck6U1/Ddm8ZyYE/Xh2KMaZ0ljF7mjUcmUO2spuTA7xuTRHGNp3kpKiSKjIQMFkxe4Ll7yV/NS1crNxvihsGgyWRn5xDqCOKmicn8myUMY7oVSxi9wKnzp9h4YiMbT25kV8kuXOoi52wOg6IGMX3g9MbaQ2q/VBxB3WyEeN05OPRXyPw6boU1e4uYOzaRWJsHyphux+8JQ0QWAv8XcAAvq+rPm33+S+BG72okkKSq/Zp8HgvkAG+rau9qL2qnGmcNO07vYOOJjWw6uYnDFYcBSIpMol9YP2JDY3np5pcC07x0tQrWg6sOxi9hx7EyTlXW8oNF4wIdlTGmBX5NGCLiAJ4HbgIKgW0islJVD1woo6qPNin/bWBqs8P8BPjYn3EC5J7NBWC4v7+oHVSVg+UH2XRiExtPbmTn6Z3Uu+sJc4QxPXk6d6TeQdbgLEb3G83X134doGckC/DcHRU5AIZdQ/aqHMKCg5g/PjnQURljWuDvGsZMoEBVDwOIyArgy8CBVsrfCzx5YUVEpgPJwHtApn9D7V7KasvYfHIzG09uZPPJzZTUlAAwpt8Ylo1bRtbgLKYlTyM8uAc/CMhZDwfXwYTbcBHEmn2nuDEtiegwayk1pjvy92/mEOB4k/VCYFZLBUVkODAS+NC7HgQ8B9wPzPdvmIHX4G5gT8mexmamA6UHUJS4sDhmD5pN1uAsrhl8Tc+pOfjiiw1QVwnjvsTWL85Scq6OJZNbmGPKGNMt+DthtDT0WVspuwx4S1Vd3vVvAWtU9fiVRlCLyHJgOcCwYcM6EGrXO37ueGMz09ZTWznfcB6HOMhIzOBbU75F1uAsJgyY0P06qjtL7ioIjYZRN5CdfZCIEAfzxl3dc7yNMV3H3wmjEBjaZD0FONlK2WXAw03WrwGuE5FvAdFAqIhUqerjTXdS1ZeAlwAyMzNbS0bdwvmG82wt2sqmk5vYdHITx84dA2BI9BAWjVzEtYOvZeagmcSGxgY40i7gdkHuGhizAGdQKO/tO8X88UlEhlpzlDHdlb9/O7cBqSIyEjiBJyl8pXkhEUkD4oHNF7ap6lebfP4gkNk8WXR3bnWTezaXTSc3sfGE55ZXp9tJRHAEMwbO4Cvjv0LW4CyGxw7vNfNQ+ez4VjhfDOO/xObDpZSer2dJhjVHGdOd+TVhqKpTRB4B1uK5rfYVVd0vIs8A21V1pbfovcAKVe3WNQRfnKk505ggthRt4WztWQDG9R/HAxMeIGtwFlOSphDq6B2z2LZbbjYEhUDqTaxefZSoUAc3pFlzlDHdmd/r/6q6BljTbNsTzdafauMYrwKvdnJonaLeVc/O4p2eZqYTm8grywOgf3h/rh18LdcOvpZrBl9DQkTPfwZGp1GFnFUw6noaQmJ4b/8pbpqQTHhIL+2rMaaXsAbjq6SqHKk80liL2H56OzXOGoKDgpmaNJXvTPsOWYOzSOufRpD0vokLO8XpfVB+FOY8yqcFZyivbmBJxuBAR+VXEwb1gX4p0+tZwvBBZX0lnxV91liLOHne028/PHY4t4+53dNZPXAmkSG9a9pzv8nJBgTGLSZ7TREx4cFcN9ZqYMZ0d5YwvFZ8eyIAtwAut4v9pfvZeHIjm05sYu+ZvbjURVRIFLMGzuIb6d/g2sHXkhKTEtige6rcbBg2m7rwAaw78Dk3TxhIWLA1RxnT3VnC8Kp31VNRX8E/ffRPbCnaQmV9JYIwccBEvpH+DbIGZ5GemE5IkE2K1yFnv/A0Sd38Uz7JP8O5Wmerg/V6UzPObxf+NtAhdIrPHvpToEPoNHYuV88SBrC7ZDd7znjm0q5pqOHGoTeSNSSL2YNmEx8eH+DoepncbM/7+CVkrz1JXEQIc8ZYc5QxPYElDGBC/wmkRKcQFxbHG4vf6HtjIrpSTjYkp1MbPZT3D+SyJGMwIQ67OcCYnsB+U4EQRwgDowYSERxhycKfqorh+Gcwfgkf5ZVwvt5lc0cZ04NYwjBdJ3c1oDBuCdl7TjIgKpRrRg0IdFTGGB9ZwjBdJzcb4kdQHZ/GBznFLJw0kGBrjjKmx7DfVtM1aivg8Mcwbgl/zTtDTYOLxTZ3lDE9iiUM0zUOvg/uBhj/JbL3nCQxJoxZI605ypiexBKG6Ro5qyAqiaqkaXyYW8yiSQNxBNkNBsb0JHZbbS/TLQeINdRCwXpI/xs+yC2hzulmyeTePXeUMb2RJQzjf4c/gvoqGPclsjcXMTA2nOnD2h4Q2S2TnzF9mM8JQ0RuA+Z6Vz9W1VX+Ccn0OrmrICyWysHX8HHeBu6bPZwga44ypsfxqQ9DRP4V+A5wwPv6B+82Y67M5fQ8ijX1Zt7PLaPe5bbBesb0UL7WMBYDU1TVDSAirwGfAz/wV2Cmlzi2GWrOeuaO2nqSIf0imDq0X6CjMsa0w9X0YfQDznqX4/wQS0BZe7mf5GaDI4yKITfwycHNfH3OSJt+xZgeyteE8a/A5yLyV0Dw9GVY7cJcmapnOpDR81h7sAqnW1lig/WM6bF8Shiq+oaIfATMwJMwvq+qp/wZmOkFinZBxXG44XFW7TzJsP6RpA/pdZVTY/oMXzu9lwLVqrpSVf8C1IrI7T7uu1BE8kSkQEQeb+HzX4rILu8rX0TKvduniMhmEdkvIntE5J6rOTHTDeRkgwRxdsg8Nh0qZUnGIGuOMqYH83Wk95OqWnFhRVXLgSfb2klEHMDzwK3ABOBeEZnQtIyqPqqqU1R1CvCfwJ+9H1UDD6jqRGAh8CsRsd7SniQ3G4Zn8d4XTlxutbmjjOnhfE0YLZXzpTlrJlCgqodVtR5YAXz5CuXvBd4AUNV8VT3oXT4JFAOJPsZrAu1MAZTkNk5lPiohqlc9ctWYvsjXhLFdRP5dREaLyCgR+SWww4f9hgDHm6wXerddRkSGAyOBD1v4bCYQChzyMV4TaLmecZ2lQ29iy2FrjjKmN/A1YXwbqAf+CLwJ1AAP+7BfS1cIbaXsMuAtVXVdcgCRQcDvgIcujANp9vlyEdkuIttLSkp8CMl0iZxsGDSF1cccuBWbO8qYXqDNZiVvP8RTqvpYO45fCAxtsp4CnGyl7DKaJSERiQVWAz9W1S0t7aSqLwEvAWRmZraWjExXqjwJJ7bDvB+TvaeI1KRoxibHBDoqY0wHtVnD8P7FP72dx98GpIrISBEJxZMUVjYvJCJpQDywucm2UOBt4HVV/d92fr8JhNzVAJwZejPbjpxlSYbVLozpDXwduPe5iKwE/hc4f2Gjqv659V1AVZ0i8giwFnAAr6jqfhF5BtiuqheSx73AClVtWkO4G88AwQEi8qB324OqusvHmE2g5GbDgDGsLIxBFZs7yphewteE0R8oBeY12aZcvAW2Vaq6BljTbNsTzdafamG/3wO/9zE+013UlMGRT+GaR8jeW8T4QbGMTowOdFTGmE7g60jvh/wdiOkl8teC20lxys3s/KCcx25JC3RExphOcsWEISL/R1X/TUT+kxbublLVf/BbZKZnylkFMYP4S3EyUG5zRxnTi7RVw8jxvm/3dyCmF6ivhoIPYOpXyd57ivQhcQwfEBXoqIwxneSKCePCU/VU9bWuCcf0aIc+BGcNpwffxO5PKvjBreMCHZExphP51IchIpnAj4DhTfdR1Qw/xWV6otxsCO/HO2UjgEMsSrfmKGN6E1/vkvoD8BiwF7hstLUxuBogbw2kLWLlvhKmDO3H0P6RgY7KGNOJfE0YJU3GTBhzuSOfQm0Fp4csYP9nlfx48fhAR2SM6WS+JownReRl4AOg7sLGtgbumT4kNxuCI/hzeRpwzKYyN6YX8jVhPASMA0K42CTl08A90we43Z7pQMbM5y8HysgcHs+guIhAR2WM6WS+JozJqpru10hMz3VyJ5wr4vSQBeTuOsdTX5rQ9j7GmB7H1+nNtzR/Up4xjXJWQVAwf6pKRwS7O8qYXsrXGsYc4Gsi8gWePgwB1G6rNahCbjY6Yg5/OlDFrJH9SYoND3RUxhg/8DVhLLzShyISr6plnRCP6WlK8qC0gNPjH+TQgfM8mDUy0BEZY/zE18kHj7ZR5ANgWsfDMT2O91Gs71RPIUgquXXSwAAHZIzxF1/7MNpiD2vuq3Ky0SGZrMhzcu3oBBKiwwIdkTHGTzorYdijUfui8uNQtIvTgxdwpLTaZqY1ppfrrIRh+iLvo1j/Ujed4CDhlonWHGVMb+Zrp3dbrEmqL8rNRhPH8buDwWSN6Ud8VGigIzLG+FG7axgi0vS5m/M7IRbTk5wvhaMbOT14AYVlNdYcZUwf0JEmqQMXFlT1bGuFRGShiOSJSIGIPN7C578UkV3eV76IlDf57GsictD7+loHYjWdLf9dUDfZ9dMJcQg3W3OUMb1eW49o/W5rHwHRrXzWdH+OllAaAAAZsElEQVQH8DxwE1AIbBORlaraNNk82qT8t4Gp3uX+wJNAJp5O9R3efW28R3eQk43GpfDKoVjmpsYRFxES6IiMMX7WVg3jZ0A8ENPsFe3DvgAzgQJVPayq9cAK4MtXKH8v8IZ3+RbgfVU9600S79PGAELTReqq4NCHFA+5iZOVdSyZbM1RxvQFbXV67wTeUdUdzT8Qkb/14fhDgONN1guBWS0VFJHhwEjgwyvsO8SH7zT+VrAeXHWsacgkNDiIBeOTAx2RMaYLtFVLOAEcFZHvtPBZpg/Hb+nuqdbGbCwD3lJV19XsKyLLRWS7iGwvKSnxISTTYTmr0MgBvPhFIjemJRITbs1RxvQFbSWMCUAU8HURiReR/hdeQIMPxy8EhjZZTwFOtlJ2GRebo3zeV1VfUtVMVc1MTEz0ISTTIc56OLiOksHzOFXlZHHG4EBHZIzpIm01Sb0IvAeMAnZw6V/96t1+JduAVBEZiae2sgz4SvNCIpKGp69kc5PNa4GfiUi8d/1m4AdtfJ/xty82QF0la10zCA8JYv64pEBHZIzpIldMGKr6H8B/iMh/q+r/c7UHV1WniDyC5+LvAF5R1f0i8gywvclzwu8FVqiqNtn3rIj8BE/SAXjmSrfvmi6SuwoNjea/j6Uwf1wyUWGdNfbTGNPd+Tpb7VUniyb7rgHWNNv2RLP1p1rZ9xXglfZ+t+lkbhfkrqF04FxO5sM/22A9Y/oUm0vK+K5wG5wvZr3OIDLUwQ1p1hxlTF9iCcP4LmcVGhTC84UjWTA+mYhQR6AjMsZ0IUsYxjfeR7GWJV/D8ZpQmzvKmD7IEobxzen9UHaED5lJTFgw16fZLczG9DWWMIxvcrNRhP8qGstNE5MJC7bmKGP6GksYxjc52VQkTOVIbbQ1RxnTR1nCMG0rOwKn9/Jx0GziIkKYM8aao4zpiyxhmLblZAPw36fHc8vEZEKD7cfGmL7IfvNN23KzqYwbR27dAJbY3FHG9FmWMMyVVRXDsS18Gjyb+MgQrhk9INARGWMCxBKGubK8NYDyUskEFk4aRIjDfmSM6avst99cWU4256OGsqt+CF+yu6OM6dMsYZjW1VbCFx+zOeQaEqLDmDXKmqOM6cssYZjWHVwHrnp+UzqRWycNwhHU0kMQjTF9hSUM07qcVdSGJbClYbQN1jPGWMIwrWiohYL1bA2dRVJsBDNG9A90RMaYALOEYVp2+COor+K18nQWpQ8iyJqjjOnz7PmapmW5q2gIjmZD7QRWWHOU6QEaGhooLCyktrY20KF0W+Hh4aSkpBASEtKu/S1hmMu5nJD3LjvCZpDoiGbq0PhAR2RMmwoLC4mJiWHEiBGIWI24OVWltLSUwsJCRo4c2a5jWJOUudzxLVBdyh8qMlicYc1Rpmeora1lwIABlixaISIMGDCgQzUwvycMEVkoInkiUiAij7dS5m4ROSAi+0Xkf5ps/zfvthwR+Q+xn4SukZONKyiUD5yTbe4o06Nc7SXinhc3c8+Lm/0UTffT0UuoXxOGiDiA54FbgQnAvSIyoVmZVOAHQJaqTgT+0bv9WiALyAAmATOA6/0Zr6HxUax7QqcxoH88GSlxgY7ImB4jOjq6cXnhwoX069ePJUuWtFj24YcfZsqUKUyYMIGIiAimTJnClClTeOutt67qO3fu3Ml7773Xobh95e8+jJlAgaoeBhCRFcCXgQNNyvwd8LyqlgGoarF3uwLhQCggQAhw2s/xmqLdUHGcPzpvZXHWYKveG9NOjz32GNXV1bz44ostfv78888DcOTIEZYsWcKuXbva9T07d+5k3759LFy4sN2x+srfTVJDgONN1gu925oaC4wVkY0iskVEFgKo6mbgr0CR97VWVXP8HK/JzcZNEOuc02ywnjEdMH/+fGJiYtq178GDB7nllluYPn06c+fOJT8/H4AVK1YwadIkJk+ezI033khNTQ3PPPMMf/jDH9pVO7la/q5htPTnqbYQQypwA5ACfCIik4AEYLx3G8D7IjJXVTdc8gUiy4HlAMOGDeu8yPuqnGxyw9KJix7ExMGxgY7GmHZ5etV+DpysbLPcgSJPGV/6MSYMjuXJL03scGy+WL58OS+//DKjR49m48aNPPLII6xbt46nn36ajz76iOTkZMrLy4mIiOCJJ55g3759/OpXv/J7XP5OGIXA0CbrKcDJFspsUdUG4AsRyeNiAtmiqlUAIvIuMBu4JGGo6kvASwCZmZnNk5G5GqWHoCSH/3U+wOLrBllzlDEBUF5ezpYtW7jzzjsbtzmdTgCysrJ44IEHuOuuu7jjjju6PDZ/J4xtQKqIjAROAMuArzQr8w5wL/CqiCTgaaI6DIwC/k5E/hVPTeV6wP8ptC/LWQXAWmcmr0y25ijTc/laE7hQs/jjN6/xZzhXRVVJSEhosU/j17/+NZ999hnZ2dlMnjyZPXv2dGlsfu3DUFUn8AiwFsgB3lTV/SLyjIjc5i22FigVkQN4+iweU9VS4C3gELAX2A3sVtVV/oy3z8vN5lBIKpFJI0hLbl/bqzGmY+Lj4xk0aBBvv/02AG63m927dwNw+PBhZs+ezU9+8hPi4+M5ceIEMTExnDt3rkti8/s4DFVdo6pjVXW0qv7Uu+0JVV3pXVZV/a6qTlDVdFVd4d3uUtVvqup472ff9XesfVplERRu4+2aqSzJsOYoYzrquuuu46677uKDDz4gJSWFtWvX+rzvihUreOGFF5g8eTITJ04kOzsbgEcffZT09HTS09NZsGABkyZNYt68eezevZupU6f2+E5v01PkrQbgPVcmL9jdUca0S1VVVePyJ5984tM+I0aMYN++fZdsGzVqVIsJZuXKlZdtS0xMZPv27VcZaftYwjAeOdmccAwhOGkcY5KsOcr0Dd2p76InsLmkDNSUoUc+YWXdNJZMtqlAjDEts4RhIH8t4nay1jWDxTZ3lDGmFZYwDOSsojRoAM6BUxiZEBXoaIwx3ZQljL6uvhp3wXpW109j8eSUtssbY/osSxh93aEPCXLWstadaXNHmb7nt4s9L+MTSxh9XW425ySamsHXMLR/ZKCjMaZH6+rpzd9++21+8YtfdDhuX9lttX2ZqwFX7rusc07h1oyhbZc3xviss6Y3dzqdBAe3fKleunRp5wTrI6th9GVHN+KoK2edawaLrTnKmE7VkenN58yZw49+9CPmzp3Lf/3Xf/GXv/yFWbNmMXXqVG6++WaKiz2PDXr55Zf5x3/8RwDuu+8+vvOd73DttdcyatSoxqlFOpPVMPqynGxqCaUyZS6D+0UEOhpjOs+7j8OpvW2XO+WdvM+XfoyB6XDrzzsW11WorKxkwwbP5NxlZWXcdtttiAgvvPACzz33HM8+++xl+xQXF7Nx40b27t3L3Xff3ek1EEsYfZXbjfPAKj5yTebmySMDHY0xpplly5Y1Lh87doy7776bU6dOUVdXx9ixY1vc5/bbb0dEyMjI4MSJE50ekyWMvurk5wSfP8U691K+n27NUaaX8bUmcKFm8dBq/8XSTlFRF8dEPfzww/zwhz9k0aJFrF+/np//vOXzCwsLa1xW7fzHA1kfRl+VuwonDspS5pEcGx7oaIwxV1BRUcGQIUNQVV577bWAxWEJo4+q27eSza7xzJvSctXWGNMxHZnevLmnnnqKpUuXcv3115OcnNyJUV4d8Ue1JVAyMzO1q6b57dFK8uD5mTzR8BDffvxZEmPC2t7HmG4uJyeH8ePHX91O3bhJyl9a+ncSkR2qmtnWvtaH0QdpzioEKB26wJKF6dv6UKLoDJYw+qDavX8h1z2GrKkZgQ7FGNODWB9GX1NRSETJHta5Z7Bw0sBAR2OM6UEsYfQxmuN5NnDp0JvpHxUa4GiMMT2J3xOGiCwUkTwRKRCRx1spc7eIHBCR/SLyP022DxORdSKS4/18hL/j7e3O736HfPcQMqfNCHQoxpgexq99GCLiAJ4HbgIKgW0islJVDzQpkwr8AMhS1TIRSWpyiNeBn6rq+yISDbj9GW+vd76UyKKtrNcv8dWJ1hxlzEPvPQTAbxf+NsCR9Az+rmHMBApU9bCq1gMrgC83K/N3wPOqWgagqsUAIjIBCFbV973bq1S12s/x9mqat4YgXJSk3ExcZEigwzGm17kwvfmuXbu45pprmDhxIhkZGfzxj3+8rGxnTG8OsHPnTt57771Oib8t/r5LaghwvMl6ITCrWZmxACKyEXAAT6nqe97t5SLyZ2AksB54XFVdTXcWkeXAcoBhw4b54xx6jYrP36FKE0jPnBvoUIzp1SIjI3n99ddJTU3l5MmTTJ8+nVtuuYV+/fo1lvF1evO27Ny5k3379rFw4cJOif1K/F3DkBa2NR8pGAykAjcA9wIvi0g/7/brgO8BM4BRwIOXHUz1JVXNVNXMxMTEzou8t6mrIrpwAx+4M1lgzVHG+NXYsWNJTU0FYPDgwSQlJVFSUuLz/gcPHuSWW25h+vTpzJ07l/z8fABWrFjBpEmTmDx5MjfeeCM1NTU888wz/OEPf2hX7eRq+buGUQg0fTJPCnCyhTJbVLUB+EJE8vAkkELgc1U9DCAi7wCzgd/4OeZeyX1wPcFaT3HKTcSGW3OU6d2e3fosuWdz2yx3ocyFvowrGdd/HN+f+f2rjmXr1q3U19czevRon/dZvnw5L7/8MqNHj2bjxo088sgjrFu3jqeffpqPPvqI5ORkysvLiYiI4IknnmDfvn386le/uurYrpa/E8Y2IFVERgIngGXAV5qVeQdPzeJVEUnA0xR1GCgH4kUkUVVLgHmAzfvRTmU7/4xoNGkzbgp0KMb0GUVFRdx///289tprBAX51qBTXl7Oli1buPPOOxu3OZ1OALKysnjggQe46667uOOOO/wS85X4NWGoqlNEHgHW4umfeEVV94vIM8B2VV3p/exmETkAuIDHVLUUQES+B3wgIgLsAH7tz3h7LWc9kUc/YLVmcuvEIYGOxhi/87Um4M+7pCorK1m8eDH/8i//wuzZs33eT1VJSEhosU/j17/+NZ999hnZ2dlMnjyZPXv2dGbIbfL71CCqugZY02zbE02WFfiu99V83/cBm7+ig1yHNxDhquL0kJuICrPZYIzxt/r6epYuXdpYG7ga8fHxDBo0iLfffpulS5fidrvZu3cvkydP5vDhw8yePZtZs2axcuVKTpw4QUxMDOfOnfPTmVzKRnr3ASXb3qJKwxk1c0mgQzGmT3jzzTfZsGEDr776auPtsldzF9SKFSt44YUXmDx5MhMnTiQ72zNDw6OPPkp6ejrp6eksWLCASZMmMW/ePHbv3s3UqVN7fKe3CTS3m6gv1vEpU7hh4tC2yxtj2q2qqgqA++67j/vuu8+nfUaMGMG+ffsu2TZq1KgWn5+xcuXKy7YlJibSVY91sITRyzmPfUaMs5RTg79FRKgj0OEY063YCO+rY01Svdypz96iXh2kzGw+wN4YY66OJYzeTJXwQ+/yGenMmTQq0NEYY3o4Sxi9WEPRPhLqT1A0aD7hIdYcZYzpGEsYvVjh5jdxqzBo1p1tFzbGmDZYwujFQg++yy4Zy8z0cYEOxZhu6ej9D3D0/gcCHUaPYQmjl6orOcyQ2oOcSJ5PWLA1RxnTFbp6evO3336bX/ziF50Wf1vsttpe6uinbzIWSJxpzVHGdLXOnN7c6XQSHNzypXrp0qWdH/wVWMLopRz5q8ljONOnTAt0KMb0OWPHjm1cbjq9edOEcSVz5szh+uuv55NPPuGOO+5g5MiR/OxnP6O+vp7ExER+//vfk5SUxMsvv9w4U+19993HgAED2LZtG6dOneK5557r9IRiCaMXqi0rYmT1Xj5MfpA0h7U6mr7n1M9+Rl1O29Ob1+Z6yvjSjxE2fhwDf/jDq46lPdObg2fywg0bNgBQVlbGbbfdhojwwgsv8Nxzz/Hss89etk9xcTEbN25k79693H333ZYwTNsKPnmTSaIMyLTmKGMCqT3Tm1+wbNmyxuVjx45x9913c+rUKerq6i6pwTR1++23IyJkZGRw4sSJDsXeEksYvZDkrqaQJNKnXRvoUIwJCF9rAhdqFsN/93qnx9De6c0viIqKalx++OGH+eEPf8iiRYtYv349P//5z1vcJywsrHHZMxF457L2il6m+txZUs/v4EjiPILt7ihjAqIj05u3pKKigiFDhqCqvPbaa50QYftYDQMoLNjH+dfvASAoNCLA0XRMaEMlw8VJv2ld/zQuY4zHhenNS0tLefXVVwEapzpvj6eeeoqlS5eSkpLCzJkzKSoq6sRofSf+qLYESmZmprZnmt+TR/IoePWbAMRF9OznXVfUNHAmKJEv//ANHK3cimdMb5STk8P48eOvah9/Nkl1Vy39O4nIDlXNbGtfu6IAg0ekMfipjwIdhjGmi/WlRNEZrA/DGGOMT/yeMERkoYjkiUiBiDzeSpm7ReSAiOwXkf9p9lmsiJwQkf/yd6zGGGNa59cmKRFxAM8DNwGFwDYRWamqB5qUSQV+AGSpapmIJDU7zE+Aj/0ZpzGmd1BVRCTQYXRbHe2z9ncNYyZQoKqHVbUeWAE0f/Tb3wHPq2oZgKoWX/hARKYDycA6P8dpjOnhwsPDKS0t9cv4g95AVSktLSU8PLzdx/B3p/cQ4HiT9UJgVrMyYwFEZCPgAJ5S1fdEJAh4DrgfmO/nOI0xPVxKSgqFhYWUlJQEOpRuKzw8nJSUlHbv7++E0VLdsHn6DwZSgRuAFOATEZkE3AesUdXjV6piishyYDnAsGHDOiFkY0xPFBISwsiRIwMdRq/m74RRCAxtsp4CnGyhzBZVbQC+EJE8PAnkGuA6EfkWEA2EikiVql7Sca6qLwEvgWcchn9OwxhjjL/7MLYBqSIyUkRCgWXAymZl3gFuBBCRBDxNVIdV9auqOkxVRwDfA15vniyMMcZ0Hb8mDFV1Ao8Aa4Ec4E1V3S8iz4jIbd5ia4FSETkA/BV4TFVL/RmXMcaYq9erpgYRkRLgaAcOkQCc6aRwAqm3nAfYuXRHveU8wM7lguGqmthWoV6VMDpKRLb7Mp9Kd9dbzgPsXLqj3nIeYOdytWxqEGOMMT6xhGGMMcYnljAu9VKgA+gkveU8wM6lO+ot5wF2LlfF+jCMMcb4xGoYxhhjfGIJowkR+YmI7BGRXSKyTkQGBzqm9hKRX4hIrvd83haRfoGOqb1E5C7v1PduEelxd7T4MsV/TyAir4hIsYjsC3QsHSUiQ0XkryKS4/3Z+k6gY2ovEQkXka0istt7Lk/77busSeoiEYlV1Urv8j8AE1T17wMcVruIyM3Ah6rqFJFnAVT1+wEOq11EZDzgBl4EvqeqV/8c3gDxTvGfT5Mp/oF7m07x31OIyFygCs+sC5MCHU9HiMggYJCq7hSRGGAHcHsP/X8RIEpVq0QkBPgU+I6qbuns77IaRhMXkoVXFJdPlNhjqOo670h7gC145vHqkVQ1R1XzAh1HO/kyxX+PoKobgLOBjqMzqGqRqu70Lp/DMxPFkMBG1T7qUeVdDfG+/HLtsoTRjIj8VESOA18Fngh0PJ3k68C7gQ6ij2ppiv8eeWHqrURkBDAV+CywkbSfiDhEZBdQDLyvqn45lz6XMERkvYjsa+H1ZQBV/ZGqDgX+gGcerG6rrXPxlvkR4MRzPt2WL+fSQ/kyxb8JEBGJBv4E/GOzFoYeRVVdqjoFT0vCTO8jIjqdv6c373ZUdYGPRf8HWA086cdwOqStcxGRrwFLgPnazTurruL/pafxZYp/EwDe9v4/AX9Q1T8HOp7OoKrlIvIRsBDo9JsT+lwN40q8zxe/4DYgN1CxdJSILAS+D9ymqtWBjqcP82WKf9PFvB3FvwFyVPXfAx1PR4hI4oW7IEUkAliAn65ddpdUEyLyJyANzx05R4G/V9UTgY2qfUSkAAgDLkwVv6UH3/G1FPhPIBEoB3ap6i2Bjcp3IrII+BWeRxC/oqo/DXBI7SIib+B5MmYCcBp4UlV/E9Cg2klE5gCfAHvx/L4D/FBV1wQuqvYRkQzgNTw/X0F4HiPxjF++yxKGMcYYX1iTlDHGGJ9YwjDGGOMTSxjGGGN8YgnDGGOMTyxhGGOM8YklDGOukohUtV3qivu/JSKjvMvRIvKiiBzyzjS6QURmiUiod7nPDa413ZclDGO6kIhMBByqeti76WU8E/qlqupE4EEgwTtR4QfAPQEJ1JgWWMIwpp3E4xfeOa/2isg93u1BIvL/eWsM2SKyRkT+xrvbV4G/eMuNBmYBP1ZVN4B3VtvV3rLveMsb0y1YddeY9rsDmAJMxjP6eZuIbACygBFAOpCEZ+rsV7z7ZAFveJcn4hm17mrl+PuAGX6J3Jh2sBqGMe03B3jDO1PoaeBjPBf4OcD/qqpbVU8Bf22yzyCgxJeDexNJvfcBP8YEnCUMY9qvpanLr7QdoAYI9y7vByaLyJV+D8OA2nbEZkyns4RhTPttAO7xPrwmEZgLbMXziMw7vX0ZyXgm7LsgBxgDoKqHgO3A097ZUxGR1AvPABGRAUCJqjZ01QkZcyWWMIxpv7eBPcBu4EPg/3iboP6E5zkY+/A8h/wzoMK7z2ouTSB/CwwECkRkL/BrLj4v40agx82eanovm63WGD8QkWhVrfLWErYCWap6yvu8gr9611vr7L5wjD8DP+jBzzM3vYzdJWWMf2R7H2oTCvzEW/NAVWtE5Ek8z/U+1trO3octvWPJwnQnVsMwxhjjE+vDMMYY4xNLGMYYY3xiCcMYY4xPLGEYY4zxiSUMY4wxPrGEYYwxxif/P2f+DK4mxhlMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x23ebd2bd160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#pd.DataFrame(grid.cv_results_).to_csv('LogisticGridSearchCV_Otto.csv')\n",
    "#cvresult = pd.DataFrame.from_csv('LogisticGridSearchCV_Otto.csv')\n",
    "#test_means = cv_results['mean_test_score']\n",
    "#test_stds = cv_results['std_test_score'] \n",
    "#train_means = cvresult['mean_train_score']\n",
    "#train_stds = cvresult['std_train_score'] \n",
    "\n",
    "\n",
    "# plot CV误差曲线\n",
    "test_means = grid.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = grid.cv_results_[ 'std_test_score' ]\n",
    "train_means = grid.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "\n",
    "# plot results\n",
    "n_Cs = len(Cs)\n",
    "number_penaltys = len(penaltys)\n",
    "test_scores = np.array(test_means).reshape(n_Cs,number_penaltys)\n",
    "train_scores = np.array(train_means).reshape(n_Cs,number_penaltys)\n",
    "test_stds = np.array(test_stds).reshape(n_Cs,number_penaltys)\n",
    "train_stds = np.array(train_stds).reshape(n_Cs,number_penaltys)\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "for i, value in enumerate(penaltys):\n",
    "    #pyplot.plot(log(Cs), test_scores[i], label= 'penalty:'   + str(value))\n",
    "    pyplot.errorbar(x_axis, test_scores[:,i], yerr=test_stds[:,i] ,label = penaltys[i] +' Test')\n",
    "    pyplot.errorbar(x_axis, train_scores[:,i], yerr=train_stds[:,i] ,label = penaltys[i] +' Train')\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'log(C)' )                                                                                                      \n",
    "pyplot.ylabel( 'f1_mirco' )\n",
    "pyplot.savefig('LogisticGridSearchCV_C.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "上图为L1和L2正则下，不同的正则参数C对应在训练集上的正确率。\n",
    "可以看出训练集上当C为1时的L1模型性能最好，当C为0时L2模型性能最好，但是在测试集上C为0时的性能最好（L1/L2均是）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 用LogisticRegressionCV实现正则化的 Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "#L2正则\n",
    "#from sklearn.linear_model import LogisticRegressionCV\n",
    "\n",
    "#Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "\n",
    "# 大量样本（6W+）、高维度（93），L2正则 --> 缺省用lbfgs，为了和GridSeachCV比较，也用liblinear\n",
    "\n",
    "#lrcv_L2 = LogisticRegressionCV(Cs=Cs, cv = 5, scoring='f1_micro', penalty='l2', solver='liblinear', multi_class='ovr')\n",
    "#lrcv_L2.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "#lrcv_L2.scores_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dict with classes as the keys, and the values as the grid of scores obtained during cross-validating each fold,\n",
    "# Each dict value has shape (n_folds, len(Cs))\n",
    "#n_Cs = len(Cs)\n",
    "#n_classes = 9\n",
    "#scores =  np.zeros((n_classes,n_Cs))\n",
    "\n",
    "#for j in range(n_classes):\n",
    "#       scores[j][:] = np.mean(lr_cv.scores_[j],axis = 0)\n",
    "    \n",
    "#mse_mean = -np.mean(scores, axis = 0)\n",
    "#pyplot.plot(np.log10(Cs), mse_mean.reshape(n_Cs,1)) \n",
    "#plt.plot(np.log10(reg.Cs)*np.ones(3), [0.28, 0.29, 0.30])\n",
    "#pyplot.xlabel('log(C)')\n",
    "#pyplot.ylabel('neg-logloss')\n",
    "#pyplot.show()\n",
    "\n",
    "#print ('C is:',lr_cv.C_)  #对多类分类问题，每个类别的分类器有一个C"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "该处对LogisticRegressionCV进行了尝试，发现得到的scores的值相较于GridSearchCV有些许不同，不明白是什么原因。\n",
    "就两种方式而言更喜欢GridSearchCV,更为方便快捷，结果一目了然。可能是因为数据集较小的原因，并没有感觉到明显的速度差异。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SVM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# default SVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "#将数据分割训练数据与测试数据(随机选择%20的数据作为测试集)\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train_part, X_val, y_train_part, y_val = train_test_split(X_train, y_train, train_size = 0.8,random_state = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "#LinearSVC不能得到每类的概率，在Otto数据集要求输出每类的概率，这里只是示意SVM的使用方法\n",
    "#https://xacecask2.gitbooks.io/scikit-learn-user-guide-chinese-version/content/sec1.4.html\n",
    "#1.4.1.2. 得分与概率\n",
    "from sklearn.svm import LinearSVC\n",
    "\n",
    "SVC1 = LinearSVC().fit(X_train_part, y_train_part)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "在线性模型中，基于LinearSVC比基于LibSVM的SVC要效率更高"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report for classifier LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
      "     verbose=0):\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.79      0.85      0.82        79\n",
      "          1       0.68      0.59      0.63        44\n",
      "\n",
      "avg / total       0.75      0.76      0.75       123\n",
      "\n",
      "\n",
      "Confusion matrix:\n",
      "[[67 12]\n",
      " [18 26]]\n"
     ]
    }
   ],
   "source": [
    "#在校验集上测试，估计模型性能\n",
    "y_predict = SVC1.predict(X_val)\n",
    "\n",
    "print(\"Classification report for classifier %s:\\n%s\\n\"\n",
    "      % (SVC1,metrics.classification_report(y_val, y_predict)))\n",
    "print(\"Confusion matrix:\\n%s\" % metrics.confusion_matrix(y_val, y_predict))"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "该处为测试所得的实验报告，可得正确率为0.75。\n",
    "对于该报告中的混淆矩阵不太会分析，但是明白视频中所讲解的对应内容。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 线性SVM正则参数调优"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "线性SVM LinearSVC的需要调整正则超参数包括C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1）\n",
    "\n",
    "这里我们用校验集（X_val、y_val）来估计模型性能"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit_grid_point_Linear(C, X_train, y_train, X_val, y_val):#自己定义函数，对C进行训练和校验\n",
    "    \n",
    "    # 在训练集是那个利用SVC训练\n",
    "    SVC2 =  LinearSVC( C = C)\n",
    "    SVC2 = SVC2.fit(X_train, y_train)\n",
    "    \n",
    "    # 在校验集上返回accuracy\n",
    "    accuracy = SVC2.score(X_val, y_val)\n",
    "    \n",
    "    print(\"accuracy: {}\".format(accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.7642276422764228\n",
      "accuracy: 0.7642276422764228\n",
      "accuracy: 0.7642276422764228\n",
      "accuracy: 0.7642276422764228\n",
      "accuracy: 0.7642276422764228\n",
      "accuracy: 0.7723577235772358\n",
      "accuracy: 0.6910569105691057\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No handles with labels found to put in legend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.7723577235772358\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X2UXXV97/H3J08Eo8EkBEQSkighhFBIYEjOFOyqtlq83gty9WJiuwS7Wq5XQi3WtvTqAgyra9nlA70L0Vuq1lKVFBGVq6lIxaLVc5KZQHhKIEkBmwGBIRAkEIhJvveP3z7kZDKTc2Yye/Z5+LzWOmtm7/Pb53wPCeeb/fv+HhQRmJmZHcq4ogMwM7Pm52RhZmZ1OVmYmVldThZmZlaXk4WZmdXlZGFmZnU5WZiZWV1OFmZmVpeThZmZ1TWh6ABGy9FHHx1z584tOgwzs5ayfv36ZyJiZr12bZMs5s6dS29vb9FhmJm1FEm/aKSdu6HMzKwuJwszM6vLycLMzOpqm5qFmVmn+/Wvf01fXx8vv/zyQc9NnjyZWbNmMXHixBG9tpOFmVmb6Ovr43Wvex1z585F0qvnI4Lt27fT19fHvHnzRvTa7oYyM2sTL7/8MjNmzDggUQBIYsaMGYPecTTKycLMrI0MTBT1zjfKycLMcrV3L3z3u3DXXUVH0p5uvBG+/OX838fJwsxy8dJL8IUvwIIF8O53w/vfDxFFR9V+rrsOvv71/N/HycLMRtXTT8NVV8EJJ8Cll8LRR6dE8cQT0NdXdHTtZdcu2LABSqX952KIjDzU+UblmiwknSvpYUlbJV0xyPPXStqQPTZL2pGdf2vN+Q2SXpb07jxjNbPDs3kzfOhDMGcOXHMNnHMO/PSnUC7D5ZenNuVysTG2m/XrYc8e6O5Ox5MnT2b79u0HJYbqaKjJkyeP+L1yGzoraTxwPfB2oA/okXRbRGystomIy2vaXwYsyc7/GFicnZ8ObAV+mFesZjZyP/sZfOYzqS4xaRJcdBF89KOp+6nq9NNh8mSoVODCC4uLtd1UKunnsmXp56xZs+jr66O/v/+gttV5FiOV5zyLpcDWiHgEQNJq4Hxg4xDtVwBXDXL+vcC/RMRLuURpZsNWLVp/5jPpbmH6dPjEJ2DlSjjmmIPbT5wIXV37v9xsdJTL8KY37f9vPnHixBHPo6gnz26o44FtNcd92bmDSJoDzAPuHOTp5cBNox6dmQ3bSy/BF78IJ58M73kPPPUUfP7z8J//CatWDZ4oqkql1G3yyitjF287i0jJotoFlbc8k8Vgg3qHqrAsB26JiL0HvIB0HPAbwO2DvoF0iaReSb2D3XaZ2eioLVp/+MMwYwbcckuqU1x6KUyZUv81urth9+5UkLXD19cHv/zlgcXtPOWZLPqA2TXHs4Anhmg71N3DhcC3I+LXg10UETdERFdEdM2cWXfvDjMbptqi9apVcPbZ+4vW73kPjB/f+GtVv9TcFTU6qoMF2iFZ9ADzJc2TNImUEG4b2EjSAmAaMNg4iRW4C8pszP3sZ3DBBam76atfhQ98AB56KNUpzjkHRjIZ+I1vhNmzPSJqtFQqadDAaaeNzfvlVuCOiD2SVpK6kMYDX4mIByWtAnojopo4VgCrY8BYL0lzSXcmnvdpNgaGKlpfeikce+zovEd3t+8sRkulkgYNTJo0Nu+X66qzEbEGWDPg3JUDjq8e4trHGKIgbmaj56WX4B//ET73Odi6NY2u+fzn4eKLG6tFDEepBDffnPrajztudF+7k7zyCtx9N1x22di9p2dwm3Wo/n64+upUj/jwh9OdxDe/Obyi9XC5bjE6NmxICWOs6hXgZGHWcapF6xNOgE9+En7zN+EnP0lf4O997/CK1sN1xhmp28TJ4vBU//uN1bBZ8OZHZh1j4EzrD3wgzbQ++eSxi+GII2DJEieLw1WppMECb3zj2L2n7yzM2tjevXDrrenu4Zxz0h3Exz8Ov/gF3HDD2CaKqlIJenrSmkY2MuXy2HZBgZOFWVs61Ezra64ZvdFNI1EqpdVS77uvuBha2S9/mZL9WHZBgbuhzNpKfz9cf316PPMMLF2aitYXXJBvLWI4ql9ylUqqYdjwrF2bfvrOwsyGrcii9XCdcAK84Q2uW4xUuZwWZlyyZGzf13cWZi3s5z+HT3+62KL1cEnpX8WeyT0y1Tuyw9iaYkQ6Plns2JH6dM1azTPPpH7/6dNT0XrlymJrEcPR3Q3f+U76DEcfXXQ0rWPPnjQ44JJLxv69Oz5ZQFoJ06zVTJ+e9l/+4AfzmUCXp2p/+9q18K53FRtLK7nvvjQ4YKzrFeBkwetfn1bRNLOxc+aZqY5SLjtZDEe1zlNEsnCB28zG3JQpaatVF7mHp1JJgwPmzBn793ayMLNClEqwbl2aOGiNqU7GG8kS8YfLycLMClEqwQsvwMaNRUfSGp55Jq0KXEQXFDhZmFlBaifnWX3VyXhjPXO7ysnCzArx5jenvbydLBpTqaRBAWeeWcz755osJJ0r6WFJWyVdMcjz10rakD02S9pR89wJkn4oaZOkjdnOeWbWJqqT85wsGlMupy1UixomnVuykDQeuB54J3AKsELSKbVtIuLyiFgcEYuB64Bba56+Efh0RCwElgJP5xWrmRWjVEo1ix076rftZHv3psEARXVBQb53FkuBrRHxSETsBlYD5x+i/QrgJoAsqUyIiDsAImJnRLyUY6xmVoDql9+6dcXG0ew2bUqDAYoqbkO+yeJ4YFvNcR9D7KktaQ4wD7gzO3USsEPSrZLukfTp7E5l4HWXSOqV1Nvf3z/K4ZtZ3s46K3VHuSvq0KrraLVrshhsJHAM0XY5cEtEVEdcTwDeAnwMOAt4E3DxQS8WcUNEdEVE18yZMw8/YjMbU1OnwqJFXlSwnkolDQY48cTiYsgzWfQBs2uOZwFPDNF2OVkXVM2192RdWHuA7wBe+d6sDXV3p2Gh+/YVHUnzqlSKm4xXlWey6AHmS5onaRIpIdw2sJGkBcA0oDzg2mmSqrcLbwM8dcesDZVK8NxzsGVL0ZE0px070iCAIrugIMdkkd0RrARuBzYBN0fEg5JWSTqvpukKYHVERM21e0ldUD+SdD+pS+vv84rVzIpT/RJ0V9TgqsX/opNFrqvORsQaYM2Ac1cOOL56iGvvAE7LLTgzawonnwxHHZW6Wi6+uOhomk+lkrqfli4tNg7P4DazQo0bB8uWeUTUUMrlNAhg6tRi43CyMLPClUpw//1pLoHtt29fKv4X3QUFThZm1gS6u9MXY29v0ZE0ly1bUvG/yJnbVU4WZla4an+8u6IOVOTOeAM5WZhZ4aZPhwULnCwGKpdT8f/kk4uOxMnCzJpEd3f6coyh1nnoQJVKKv6Pa4Jv6iYIwcwsdbX098OjjxYdSXPYuTMV/ZuhCwqcLMysSVS/FN0VlfT0pKK/k4WZWY1Fi9LGPp7JnVST5rJlxcZR5WRhZk1hwoQ0Ksp3Fkmlkor+06cXHUniZGFmTaNUgg0bYNeuoiMpVkS6w2qWLihwsjCzJlIqwZ49sH590ZEU69FHU7HfycLMbBAucifVz98MM7ernCzMrGkccwy86U1OFuVyKvYvWlR0JPs5WZhZUymVPDmvUkn7k0/IdROJ4ck1WUg6V9LDkrZKumKQ56+VtCF7bJa0o+a5vTXPHbTDnpm1p+5ueOIJ6OsrOpJi7NqVivzN1AUFOW5+JGk8cD3wdtKe2j2SbouIV7dHjYjLa9pfBiypeYldEbE4r/jMrDnV1i1mzy42liKsX5+K/M1U3IZ87yyWAlsj4pGI2A2sBs4/RPsVwE05xmNmLeC002Dy5M6tWzTTSrO18kwWxwPbao77snMHkTQHmAfcWXN6sqReSRVJ784vTDNrJpMmQVdX587krlRSkf+YY4qO5EB5JgsNcm6oktVy4JaI2Ftz7oSI6ALeD/ytpDcf9AbSJVlC6e3v7z/8iM2sKZRKcPfd8MorRUcy9iqV5rurgHyTRR9Q2+M4C3hiiLbLGdAFFRFPZD8fAf6NA+sZ1TY3RERXRHTNnDlzNGI2syZQKqVEce+9RUcytrZtg8cf77xk0QPMlzRP0iRSQjhoVJOkBcA0oFxzbpqkI7LfjwbOBjYOvNbM2lP1y7LTuqKacTJeVW7JIiL2ACuB24FNwM0R8aCkVZLOq2m6AlgdccCo6oVAr6R7gR8Dn6odRWVm7e3449NIqE4rclcqqbh/2mlFR3KwXKd8RMQaYM2Ac1cOOL56kOt+DvxGnrGZWXMrlTovWZTLcOaZqcjfbDyD28yaUqkEjz0GTz5ZdCRj45VXUlG/GesV4GRhZk2q2m/fKXcX996bEkYz1ivAycLMmtSSJTBxYucki2ox33cWZmbDMHlyShidMiKqUoFZs1Jxvxk5WZhZ0+ruhp6etFZSu6tUmrcLCpwszKyJlUppFdb77y86knw9+WQq5jdrFxQ4WZhZE+uUnfOadfHAWk4WZta05syBN7yh/esWlUoq5p9xRtGRDM3JwsyaltQZk/PK5VTMnzy56EiG5mRhZk2tVIItW2D79qIjyceePamI38xdUOBkYWZNrt3rFvffn4r4zTwSCpwszKzJdXXB+PHtmyxaobgNThZm1uSmTEmrsLZrsiiX4dhjUzG/mTlZmFnTK5Vg7VrYu7d+21ZTnYynwfYWbSJOFmbW9Lq74YUXYNOmoiMZXdu3p+J9s3dBQc7JQtK5kh6WtFXSFYM8f62kDdljs6QdA56fKulxSZ/PM04za27tWuRulXoF5JgsJI0HrgfeCZwCrJB0Sm2biLg8IhZHxGLgOuDWAS9zDXBXXjGaWWs48USYMaP9JudVKql439VVdCT15XlnsRTYGhGPRMRuYDVw/iHarwBuqh5IOhM4FvhhjjGaWQto18l5lUoq3k+ZUnQk9eWZLI4HttUc92XnDiJpDjAPuDM7Hgd8FvjzHOMzsxZSKsHGjbBjR/22rWDv3lS0b4UuKGgwWUj6lqR3ZV/ijRqsth9DtF0O3BIR1bEOHwbWRMS2IdpX47pEUq+k3v7+/mGEZmatpvql2tNTbByjZdOmVLRvq2QBfBF4P7BF0qckndzANX3A7JrjWcATQ7RdTk0XFNANrJT0GPAZ4AOSPjXwooi4ISK6IqJr5syZDYRkZq1q6dLUHdUudYtql1qzz9yumtBIo4j4V+BfJR1Fqi3cIWkb8PfA1yLi14Nc1gPMlzQPeJyUEN4/sJGkBcA04NW/AhHx+zXPXwx0RcRBo6nMrHNMnQqLFrVP3aJchunTU/G+FTTcrSRpBnAx8EfAPcD/Ac4A7hisfUTsAVYCtwObgJsj4kFJqySdV9N0BbA6IobqojIzA/YXudvh26JSSZ+n2SfjVTV0ZyHpVuBk4J+A/xYRv8ye+mdJvUNdFxFrgDUDzl054PjqQ713RHwV+GojcZpZe+vuhi99CTZvhgULio5m5HbsSMX6FSuKjqRxDSUL4PMRcedgT0REC4wQNrN2UDs5r5WTxbp16WerFLeh8W6ohZJeXz2QNE3Sh3OKycxsUCefnGoXrV63qFRS99PSpUVH0rhGk8UfR8Sro5sj4jngj/MJycxscOPGwbJlrT8iqlJJxfqpU4uOpHGNJotx0v4yTLaUx6R8QjIzG1p3d9owaOfOoiMZmYj9xe1W0miyuB24WdLvSHobaU7ED/ILy8xscKUS7NsHvUMOrWlumzfDc8+1b7L4S9JSHP8LuBT4EfAXeQVlZjaUZcvSz1btimqllWZrNTopbx9pFvcX8w3HzOzQpk9PI6FatchdqaRaxcKFRUcyPI2uDTVf0i2SNkp6pPrIOzgzs8G08uS8cjndHY1rsa3nGg33H0h3FXuAtwI3kibomZmNuVIJnn4aHnus6EiGZ+fOVJxvtS4oaDxZHBkRPwIUEb/IZl2/Lb+wzMyGVl18r9XqFr29qTjfKosH1mo0WbycLU++RdJKSRcAx+QYl5nZkBYtShsGtVrdoprcWmkyXlWjyeJPgdcAfwKcCfwBcFFeQZmZHcqECXDWWa2XLCoVOOmktEVsq6mbLLIJeBdGxM6I6IuID0bEeyKixf6YzKyddHfDPffArl1FR9KY6mS8VuyCggaSRbZ73Zm1M7jNzIpWKsGePXD33UVH0phHH01F+VYsbkPjq87eA3xX0jeBF6snI+LWXKIyM6ujOjmvUoGzzy42lka06mS8qkaTxXRgOweOgArAycLMCnHssTBvXuuMiKpUUlH+1FOLjmRkGp3B/cGRvLikc0k76o0HvhQRnxrw/LWkeRuQCujHRMTrJc0hJaLxwETguoj4vyOJwczaV3c33HVX0VE0plxORfkJjf4Tvck0ulPeP5DuJA4QEX94iGvGA9cDbwf6gB5Jt0XExprrL69pfxmwJDv8JfCbEfGKpNcCD2TXPtFIvGbWGUol+MY3oK8PZs0qOpqh7doFGzbAxz5WdCQj1+jQ2e8B388ePwKmAvUWCF4KbI2IRyJiN7AaOP8Q7VeQVrMlInZHxCvZ+SOGEaeZdZBq/3+zd0XdfXcqxrdqvQIa/BKOiG/VPL4OXAjU63k7HthWc9yXnTtI1u00j7SybfXcbEn3Za/xN4PdVUi6RFKvpN7+/v5GPoqZtZHTT4fJk5t/vkWrF7dh5P9inw+cUKfNYENth1r2azlwSzZMNzWM2BYRpwEnAhdJOvagF4u4ISK6IqJr5syZDYZuZu1i0iQ488zmTxblcirGH3vQt1jraHTV2Rck/ar6AP4faY+LQ+kDZtcczwKGqjksJ+uCGii7o3gQeEsjsZpZZymVYP162L276EiG1oo74w3UaDfU6yJias3jpIj4Vp3LeoD5kuZJmkRKCLcNbCRpATANKNecmyXpyOz3acDZwMONfSQz6yTd3fDKK6mA3Iz6+uDxx1t35nZVo3cWF0g6qub49ZLefahrImIPsJK0Jesm4OaIeFDSKknn1TRdAayOOGBl+oXAWkn3AncBn4mI+xv7SGbWSar/Ym/Wrqhq8b3V7ywUDeweImlDRCwecO6eiFgy1DVjraurK3pbdVNeMzsss2fDW96ShtE2mz/7M/jCF+D551ONpdlIWh8RXfXaNVrgHqxdi04tMbN2093dvMNnK5VUhG/GRDEcjSaLXkmfk/RmSW/KZl6vzzMwM7NGlUpp17wnnyw6kgPt3p2K763eBQWNJ4vLgN3APwM3A7uAS/MKysxsOKpfxmvXFhvHQBs2pOJ7OySLRteGehG4IudYzMxG5IwzYOLE1BV1/qHWiRhj1aJ7q4+EgsZHQ90h6fU1x9Mk3Z5fWGZmjZs8GZYsab4RUeVyWrPq+EHXrmgtjXZDHR0RO6oHEfEc3oPbzJpIqQQ9PWkNpmbRDpPxqhpNFvskvbq8h6S5DL10h5nZmCuV4KWX4P4mmZH15JOp6N4uyaLR4a8fB/5dUnXl+N8CLsknJDOz4avWBSqV1CVVtHaqV0Djy338AOgiLbnxz8CfkUZEmZk1hTlz0kJ9zVK3qFRS0b0ZEtdoaHTzoz8CPkJaDHADUCKt5fS2Q11nZjZWpNTl00zJYvFiOPLIoiMZHY3WLD4CnAX8IiLeStrRzhtImFlT6e6GzZth+/Zi49izJxXb26ULChpPFi9HxMsAko6IiIeABfmFZWY2fM0yOe/++1OxvV2K29B4sujL5ll8B7hD0ncZem8KM7NCdHXB+PHFd0W1w854AzU6g/uC7NerJf0YOAr4QW5RmZmNwJQpcNppxS8qWKmkYvvcucXGMZqGvXJsRNxVv5WZWTFKJfja12Dv3nSXUYRyOcWhwTaXblEj3YO7IZLOlfSwpK2SDlpbStK1kjZkj82SdmTnF0sqS3pQ0n2S3pdnnGbWPkoleOEFeOihYt5/+3bYsqW9uqAgxz0pJI0HrgfeTtqPu0fSbRGxsdomIi6vaX8ZaZQVwEvAByJii6Q3Ausl3V675IiZ2WCqI5DKZVi0aOzfv1pcb6eRUJDvncVSYGtEPBIRu4HVwKHWg1wB3AQQEZsjYkv2+xPA08DMHGM1szZx4okwfXpxRe5yGcaNS8X2dpJnsjge2FZz3JedO4ikOcA84M5BnlsKTAL+I4cYzazNFD05r1JJRfYpU4p5/7zkmSwGK+0MtfjgcuCWiNh7wAtIxwH/BHwwIvYd9AbSJZJ6JfX293uOoJklpRJs3Jj2vR5Le/embqh2q1dAvsmiD5hdczyLoedmLCfrgqqSNBX4PvCJiBj03wgRcUNEdEVE18yZ7qUys6S7GyJg3bqxfd9Nm1Jxvd3qFZBvsugB5kuaJ2kSKSHcNrCRpAXANNJaU9Vzk4BvAzdGxDdzjNHM2tBZZ6XuqLHuimrHyXhVuSWLiNgDrARuBzYBN0fEg5JWSTqvpukKYHVE1HZRXUhaBv3imqG1i/OK1czay1FHwSmnFJMspk+H+fPH9n3HQm5DZwEiYg2wZsC5KwccXz3IdV8DvpZnbGbW3rq74dZbU3fUWE2Oq+6M106T8apynZRnZlaUUgmefTZNkBsLzz+fiurt2AUFThZm1qaqX9pj1RW1bl26i3GyMDNrIQsXwtSpY7eoYKWSup+WLh2b9xtrThZm1pbGjYNly8buzqJcTkX1o44am/cba04WZta2SiW47z548cV83ydif3G7XTlZmFnb6u6GffvSFqd52rIFnnuuPSfjVTlZmFnbqtYP8u6KqtZFfGdhZtaCZsyAk07KP1lUKqmYvnBhvu9TJCcLM2trpVL6l38MtYzpKKhUUjF9XBt/o7bxRzMzS3WEp5+Gxx7L5/V37kxF9HbuggInCzNrc3lPzuvtTUV0JwszsxZ26qlpI6K8kkX1dZcty+f1m4WThZm1tQkT0pLlec3kLpdTEX3GjHxev1k4WZhZ2yuV4J57YNeu0X3dTpiMV+VkYWZtr1SCPXtSwhhNjz2WiudOFodJ0rmSHpa0VdIVgzx/bc3mRpsl7ah57geSdkj6Xp4xmln7q36Zj3ZXVLVe0c4zt6ty2/xI0njgeuDtpP24eyTdFhEbq20i4vKa9pcBS2pe4tPAa4D/mVeMZtYZjj0W5s0b/SJ3uQyveU0qore7PO8slgJbI+KRiNgNrAbOP0T7FcBN1YOI+BHwQo7xmVkHKZVGP1lUKql4PiHXPUebQ57J4nhgW81xX3buIJLmAPOAO3OMx8w6WHc39PWlx2jYtSvVQDqhCwryTRaD7UI71IT75cAtEbF3WG8gXSKpV1Jvf3//sAM0s84x2pPz7r47Fc07obgN+SaLPmB2zfEs4Ikh2i6npguqURFxQ0R0RUTXzJkzRxCimXWK00+HI44YvWRRfR0ni8PXA8yXNE/SJFJCuG1gI0kLgGnAGG1+aGadaNIkOPPM0RsRVamkovmxx47O6zW73JJFROwBVgK3A5uAmyPiQUmrJJ1X03QFsDriwDUhJf0U+CbwO5L6JP1eXrGaWWfo7ob162H37sN/rXK5c+4qIMehswARsQZYM+DclQOOrx7i2rfkF5mZdaJSCT77Wbj33jSKaaT6+uDxxzsrWXgGt5l1jNEqcndavQKcLMysg8yalR6HW7col1OxfPHi0YmrFThZmFlHGY3JeZVKKpZPmjQ6MbUCJwsz6yilEjz6KDz11Miu3707Fck7qQsKnCzMrMNUZ1yP9O5iwwZ45ZXOmbld5WRhZh1lyRKYOHHkyaITi9vgZGFmHebII1Nh+nCSxfHHp0J5J3GyMLOO090N69altZ2Gq1LpvC4ocLIwsw5UKsFLL8EDDwzvuqeeSsXxTuuCAicLM+tAI52c16n1CnCyMLMONHduWgBwuJPzKpVUHD/jjFzCampOFmbWcaSRTc4rl1Nx/Mgj84mrmTlZmFlHKpVg82bYvr2x9nv2QE9PZ3ZBgZOFmXWo6pf+unWNtX/ggVQUd7IwM+sgZ50F48Y1XreotuvEYbPgZGFmHWrKFDjttMbrFpUKHHNMKo53olyThaRzJT0saaukKwZ5/lpJG7LHZkk7ap67SNKW7HFRnnGaWWcqlWDtWti3r37bSiW1l/KPqxnlliwkjQeuB94JnAKskHRKbZuIuDwiFkfEYuA64Nbs2unAVcAyYClwlaRpecVqZp2puxt+9SvYtOnQ7bZvT8XwTu2CgnzvLJYCWyPikYjYDawGzj9E+xXATdnvvwfcERHPRsRzwB3AuTnGamYdqNHJeWvXHti+E+WZLI4HttUc92XnDiJpDjAPuHM410q6RFKvpN7+/v5RCdrMOsf8+TB9ev1kUamkYnhX19jE1YzyTBaD9ezFEG2XA7dExN7hXBsRN0REV0R0zZw5c4Rhmlmnqk7OqzciqlxOxfDXvnZs4mpGeSaLPmB2zfEs4Ikh2i5nfxfUcK81MxuxUgk2boTnnx/8+b17UzdUJ3dBQb7JogeYL2mepEmkhHDbwEaSFgDTgNrcfjvwDknTssL2O7JzZmajqlSCiDQ7ezAPPQQvvOBkkVuyiIg9wErSl/wm4OaIeFDSKknn1TRdAayOiKi59lngGlLC6QFWZefMzEbV0qWpO2qorqhqPaOTR0IBTMjzxSNiDbBmwLkrBxxfPcS1XwG+kltwZmbAUUfBKacMXeQul2HatFQM72SewW1mHa+6Am0MMgSn0yfjVTlZmFnHK5Xg2Wdh69YDzz//fCp+d3q9ApwszMxerUcMrFusW5fuNjq9XgFOFmZmLFwIU6ceXLeoVFL309KlxcTVTJwszKzjjRuXEsJgyWLhwlQE73ROFmZmpK6m++6DF19MxxEpWbgLKnGyMDMjFbH37oXe3nS8ZUsqeru4nThZmJkBy5aln9WuqOpPJ4vEycLMDJgxA046af+IqHI5Fb1POeXQ13UKJwszs0zt5LxKJRW9x/lbEnCyMDN7VakETz2VJuLdd5+7oGo5WZiZZaojn667Lu3L7ZFQ+zlZmJllTj0VXvMauPHGdFwtepuThZnZqyZMgLPOgl270iqzM2YUHVHzcLIwM6tRrVO4XnGgXJOFpHMlPSxpq6QrhmhzoaSNkh6U9I2a838j6YHs8b484zQzq6rWKVyvOFBumx9JGg9cD7ydtKd2j6TbImJjTZv5wF8BZ0fEc5Luxk1YAAAGn0lEQVSOyc6/CzgDWAwcAdwl6V8i4ld5xWtmBvCOd8BHPwoXXlh0JM0lzzuLpcDWiHgkInYDq4HzB7T5Y+D6iHgOICKezs6fAtwVEXsi4kXgXuDcHGM1MwPgyCPhs591vWKgPJPF8cC2muO+7Fytk4CTJP1MUkVSNSHcC7xT0mskHQ28FZg98A0kXSKpV1Jvf39/Dh/BzMwg3z24B9uEcOCmhROA+cBvA7OAn0o6NSJ+KOks4OdAP1AG9hz0YhE3ADcAdHV1DbIhopmZjYY87yz6OPBuYBbwxCBtvhsRv46IR4GHScmDiPjriFgcEW8nJZ4tOcZqZmaHkGey6AHmS5onaRKwHLhtQJvvkLqYyLqbTgIekTRe0ozs/GnAacAPc4zVzMwOIbduqIjYI2klcDswHvhKRDwoaRXQGxG3Zc+9Q9JGYC/w5xGxXdJkUpcUwK+AP4iIg7qhzMxsbCiiPbr6u7q6ore6a4mZmTVE0vqI6KrXzjO4zcysLicLMzOrq226oST1A784jJc4GnhmlMIpUrt8DvBnaVbt8lna5XPA4X2WORExs16jtkkWh0tSbyP9ds2uXT4H+LM0q3b5LO3yOWBsPou7oczMrC4nCzMzq8vJYr8big5glLTL5wB/lmbVLp+lXT4HjMFncc3CzMzq8p2FmZnV5WSRkXSNpPskbZD0Q0lvLDqmkZL0aUkPZZ/n25JeX3RMIyXpf2S7KO6T1HIjVxrZLbJVSPqKpKclPVB0LIdD0mxJP5a0Kfu79ZGiYxopSZMlrZN0b/ZZPpnbe7kbKpE0tboTn6Q/AU6JiA8VHNaISHoHcGe2PtffAETEXxYc1ohIWgjsA/4O+FhEtMyaLtlukZup2S0SWFG7W2QrkfRbwE7gxog4teh4RkrSccBxEXG3pNcB64F3t+Kfi9ICelMiYqekicC/Ax+JiMpov5fvLDIDtmydwsF7b7SMiPhhzcKLFdLy8C0pIjZFxMNFxzFCjewW2TIi4ifAs0XHcbgi4pcRcXf2+wvAJg7emK0lRLIzO5yYPXL57nKyqCHpryVtA34fuLLoeEbJHwL/UnQQHaqR3SKtQJLmAkuAtcVGMnLZlg4bgKeBOyIil8/SUclC0r9KemCQx/kAEfHxiJgNfB1YWWy0h1bvs2RtPk7aYfDrxUVaXyOfpUU1slukFUTSa4FvAX86oGehpUTE3ohYTOpBWCoply7CPLdVbToR8bsNNv0G8H3gqhzDOSz1Pouki4D/CvxONHlhahh/Lq2mkd0irQBZ//63gK9HxK1FxzMaImKHpH8DzgVGfRBCR91ZHIqk+TWH5wEPFRXL4ZJ0LvCXwHkR8VLR8XSwRnaLtDGWFYW/DGyKiM8VHc/hkDSzOtpR0pHA75LTd5dHQ2UkfQtYQBp58wvgQxHxeLFRjYykrcARwPbsVKWFR3ZdAFwHzAR2ABsi4veKjapxkv4L8Lfs3y3yrwsOacQk3QT8NmmF06eAqyLiy4UGNQKSzgF+CtxP+v8d4H9HxJriohqZbNvpfyT9/RoH3BwRq3J5LycLMzOrx91QZmZWl5OFmZnV5WRhZmZ1OVmYmVldThZmZlaXk4XZMEjaWb/VIa+/RdKbst9fK+nvJP1HtmLoTyQtkzQp+72jJs1ac3OyMBsjkhYB4yPikezUl0gL882PiEXAxcDR2aKDPwLeV0igZoNwsjAbASWfztawul/S+7Lz4yR9IbtT+J6kNZLem132+8B3s3ZvBpYBn4iIfQDZ6rTfz9p+J2tv1hR8m2s2Mv8dWAycTprR3CPpJ8DZwFzgN4BjSMtffyW75mzgpuz3RaTZ6HuHeP0HgLNyidxsBHxnYTYy5wA3ZSt+PgXcRfpyPwf4ZkTsi4gngR/XXHMc0N/Ii2dJZHe2OY9Z4ZwszEZmsOXHD3UeYBcwOfv9QeB0SYf6f/AI4OURxGY26pwszEbmJ8D7so1nZgK/BawjbWv5nqx2cSxp4b2qTcCJABHxH0Av8MlsFVQkza/u4SFpBtAfEb8eqw9kdihOFmYj823gPuBe4E7gL7Jup2+R9rF4gLRv+Frg+eya73Ng8vgj4A3AVkn3A3/P/v0u3gq03Cqo1r686qzZKJP02ojYmd0drAPOjogns/0GfpwdD1XYrr7GrcBftfD+49ZmPBrKbPR9L9uQZhJwTXbHQUTsknQVaR/u/xzq4myjpO84UVgz8Z2FmZnV5ZqFmZnV5WRhZmZ1OVmYmVldThZmZlaXk4WZmdXlZGFmZnX9f1leXnjIp0vRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x23ebd008080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#需要调优的参数\n",
    "C_s = np.logspace(-3, 3, 8)# logspace(a,b,N)把10的a次方到10的b次方区间分成N份  \n",
    "#penalty_s = ['l1','l2']\n",
    "\n",
    "accuracy_s = []\n",
    "for i, oneC in enumerate(C_s):\n",
    "#    for j, penalty in enumerate(penalty_s):\n",
    "#对于每一个C都跑一遍校验集上的性能\n",
    "    tmp = fit_grid_point_Linear(oneC, X_train, y_train, X_val, y_val)\n",
    "    accuracy_s.append(tmp)\n",
    "\n",
    "x_axis = np.log10(C_s)\n",
    "#for j, penalty in enumerate(penalty_s):\n",
    "pyplot.plot(x_axis, np.array(accuracy_s), 'b-')\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'log(C)' )                                                                                                      \n",
    "pyplot.ylabel( 'accuracy' )\n",
    "pyplot.savefig('SVM_Otto.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "可以看出最佳的性能点为accuracy: 0.7723577235772358"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "相较于Logistic回归，有以下三点感受：\n",
    "1.我可以很直观的感觉到线性SVM运行速度较慢\n",
    "2.SVM在参数不同的情况下最优解也有一定的差异，而Logistic回归就不存在这个问题\n",
    "3.SVM不需要认为添加正则项，而Logistic回归则需要程序员额外添加正则项。\n",
    "综上所述，就我个人而言更喜欢使用线性SVM，因为对于经验欠缺的我来说，添加正则项需要多次尝试才能确定。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RBF核SVM正则参数调优"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "RBF核是SVM最常用的核函数。 RBF核SVM 的需要调整正则超参数包括C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和核函数的宽度gamma C越小，决策边界越平滑； gamma越小，决策边界越平滑。\n",
    "\n",
    "这里我们用校验集（X_val、y_val）来估计模型性能"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit_grid_point_RBF(C, gamma, X_train, y_train, X_val, y_val):#自己定义函数，对C进行训练和校验\n",
    "    \n",
    "    # 在训练集是那个利用SVC训练\n",
    "    SVC3 =  SVC( C = C, kernel='rbf', gamma = gamma)\n",
    "    SVC3 = SVC3.fit(X_train, y_train)\n",
    "    \n",
    "    # 在校验集上返回accuracy\n",
    "    accuracy = SVC3.score(X_val, y_val)\n",
    "    \n",
    "    print(\"accuracy: {}\".format(accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.7804878048780488\n",
      "accuracy: 0.8130081300813008\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.6422764227642277\n",
      "accuracy: 0.7886178861788617\n",
      "accuracy: 0.7642276422764228\n",
      "accuracy: 0.9186991869918699\n",
      "accuracy: 1.0\n",
      "accuracy: 1.0\n",
      "accuracy: 1.0\n",
      "accuracy: 0.7642276422764228\n",
      "accuracy: 0.7886178861788617\n",
      "accuracy: 0.991869918699187\n",
      "accuracy: 1.0\n",
      "accuracy: 1.0\n",
      "accuracy: 1.0\n",
      "accuracy: 0.7560975609756098\n",
      "accuracy: 0.8130081300813008\n",
      "accuracy: 1.0\n",
      "accuracy: 1.0\n",
      "accuracy: 1.0\n",
      "accuracy: 1.0\n"
     ]
    }
   ],
   "source": [
    "#需要调优的参数\n",
    "C_s = np.logspace(-3, 3, 6)# logspace(a,b,N)把10的a次方到10的b次方区间分成N份 \n",
    "gamma_s = np.logspace(-3, 3, 6)  \n",
    "\n",
    "accuracy_s = []\n",
    "for i, oneC in enumerate(C_s):\n",
    "    for j, gamma in enumerate(gamma_s):\n",
    "        tmp = fit_grid_point_RBF(oneC, gamma, X_train, y_train, X_val, y_val)\n",
    "        accuracy_s.append(tmp)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "结果相较于线性SVM有了明显改善，说明gamma参数设置较为合理,该处尝试了多次参数修改，现在输出相对合理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXd4lFXah+8z6SQhCR0SQgiEkoSQhNARI6KiNEUURXexACrYP1y7IoqrKK5lkQVFaStYsKCLIrgIKiAgAWkuEARSgJDe28zz/fFOhgQSmEAmk8C5r2uuzJzynufMZN5nnlN+R4kIGo1Go9GcDZOzDdBoNBpNw0c7C41Go9GcE+0sNBqNRnNOtLPQaDQazTnRzkKj0Wg050Q7C41Go9GcE+0sNBqNRnNOtLPQaDQazTnRzkKj0Wg058TV2QbUFS1atJCQkBBnm6HRaDSNit9++y1dRFqeq9xF4yxCQkLYtm2bs83QaDSaRoVS6og95fQwlEaj0WjOiXYWGo1Gozkn2lloNBqN5pxcNHMW1VFWVkZycjLFxcXONkWj0Vwgnp6eBAUF4ebm5mxTLkkuameRnJyMr68vISEhKKWcbY5GozlPRISMjAySk5Pp2LGjs825JHHYMJRS6gOlVJpSancN+Uop9bZS6qBS6nelVGylvAlKqQPWx4TztaG4uJjmzZtrR6HRNHKUUjRv3lyPEjgRR85ZLASGnSX/WiDM+pgMzAVQSjUDngf6An2A55VSAedrhHYUGs3Fgf4uOxeHDUOJyAalVMhZiowGFotxrutmpZS/UqotEA+sEZFMAKXUGgyns8xRtmo0FyOlBYXkpWc624w6pSAji3/f96SzzXAKIoIFCxaxIGL8NR6Cu68Xd745y6HtO3POIhBIqvQ62ZpWU/oZKKUmY0QlBAcHO8bKOqJv376UlJSQmZlJUVERgYFGl7788ktqs/P8888/Jzw8nG7dutWq/UGDBvHPf/6T6OjoWtWr4PXXX6ddu3aMHz/+vOrXBzfddBOvvvoqoaGhZ+Rt2rSJe++9F6UUIsKLL77IqFGjziiXmJjIrbfeSmZmJn369GHRokWNdkI192QG5eaLa9jGbCnjeOYuZ5vR4HApaObwNpzpLKqLKeUs6WcmiswH5gPExcVVW6ah8OuvvwKwcOFCtm3bxj//+c/zus7nn3+OyWSqtbO4EMrKyliyZAnbt2+vtzbPh3vvvZfXXnuNuXPnnpHXs2dPfvvtN1xdXUlNTSUmJoYRI0ZgMlUdiX3sscf429/+xtixY5k4cSILFy5k0qRJ9dWFOkNEKDeXYlLutArt4Gxz6oys0nL+7+NvnG2GjaLyInJKcsgpySG7JJvskuxqn1cuk1uai0Us1V7PpEw0dW+Kv4c/fh5+Vf5WPD/9tb+HP56ung7vqzOdRTLQvtLrICDVmh5/WvqP9WaVE/j222+ZMWMGJSUlhIWF8cEHH+Dt7c1jjz3Gf/7zH1xdXbn22msZMWIEq1at4pdffmH69Om1jkoqWLp0Ka+++ioiwqhRo3j55ZcBmDdvHrNnz6Zdu3Z07twZHx8f3nzzTdasWUPv3r1xcXEBYPPmzUyePBkfHx8GDhzImjVr2LFjB4mJidxxxx3k5+djMpl499136du3L2vXrmXmzJk0b96cnTt3Mm7cOLp06cI777xDSUkJK1euJCQkhNtvvx0/Pz/27t3L0aNH+fDDD1mwYAG//vorAwcOZMGCBQBMnjyZ7du3U1RUxLhx43juuecAiI+PZ+LEiZjNZputFTRp0sT2vKioCDBuqJUxm81s2LCBTz/9FIAJEybwyiuvNEpnUZSVC1hw9/BwtimNArPFTG5p7hk3+Jpu/hXPS8wlNV7Ty9Wryk29jXebam/+lZ/7uvtiUg1z+5szncVK4H6l1HKMyewcETmmlFoNvFxpUvtq4IIHKV/4eg97U3Mv9DJVCG/XlOdHRlzQNdLS0njllVf44YcfaNKkCTNnzuStt97i7rvvZtWqVezZswelFNnZ2fj7+3PdddcxduxYrr/++vNqLzk5mWeeeYZt27bh5+fH0KFD+eabb+jZsyevvPIK27dvx9vbm/j4ePr06QPAL7/8Qq9evWzXuPPOO1m0aBF9+vRh2rRptvS2bduyZs0aPD09+eOPP5gwYYItotq5cyf79u3Dz8+PkJAQpkyZwtatW5k9ezb//Oc/ef311wHIyclh3bp1rFixgpEjR7Jp0ya6detGbGwsu3fvJjIykldeeYVmzZpRXl7OFVdcwdixYwkPD8fFxYWQkBB2795Nz549z+j7xo0bmTRpEkeOHOGjjz46w6GcPHmSFi1a2NKDgoJISUk5r/fZ2RTl5QHg3ey814Y0WgrLCmv1Sz+7JJu80jyk+gEMXJRLlV/07XzaEd48vNpf/5Vv/u4u7vXcc8fiMGehlFqGESG0UEolY6xwcgMQkX8Bq4DrgINAIXCnNS9TKfUisNV6qRkVk90XIxs3bmTv3r0MGDAAgNLSUgYNGkSzZs0wmUxMmjSJ4cOHM2LEiDpp79dff2XIkCG0aNECgPHjx7NhwwaKi4sZMmQIAQHGzWXs2LEcPXoUgGPHjhETEwNAeno6paWlNkcyfvx41q5dC0BJSQn3338/O3fuxNXVlcTERFu7ffv2pXXr1gCEhoZyzTXXANCjRw82bdpkKzdy5Ehbert27QgPDwcgPDycw4cPExkZybJly1iwYAHl5eWkpqayd+9eW7lWrVqRmpparbMYMGAAe/bsYc+ePdx1110MGzYMd/dTX+jTIw1ovCtwys2lKOWKm9elE1kcyz/GC5te4JfUX2os4+3mXeWmHugbiJ+7H/6e1f/S9/fwx8fNp9H+H9QljlwNdes58gWYWkPeB8AHdWnPhUYAjkJEGDZsGEuWLDkjb9u2baxZs4bly5czd+5cvv/++xqvU/kGPmbMGNvQTHXt1SYdwMvLy7a+/WzlZs+eTfv27Vm6dCllZWX4+PjY8jwqDYeYTCbba5PJRHl5+RnlKpepXO7AgQO89dZbbNmyBX9/f26//fYqa++Li4vx8vLis88+46WXXgKMeaLKE/sRERG4u7uzd+/eKumtWrUiPT3dNoyVnJxMu3btauxvQ6UkrwARM+5u3s42pV4QET7d/ymzt80GYHLUZAJ9Ao1owN168/f0x8/dDzeXxrlYoSFwUe/gbgwMGDCAhx56iEOHDhEaGkpBQQGpqam0adOG4uJiRowYQd++fW2/nH19fcmzDjFUxt3dnR07dpyzvX79+vHYY4+RkZGBn58fy5cvZ9q0afTo0YPHH3+c7OxsvL29+fzzz4mLiwOge/fuHDx4EICWLVvi5ubGtm3biIuLY/ny5bZr5+Tk0LlzZ5RSLFq06KyO5XzJzc3F19eXpk2bcuzYMVavXs2wYae28xw4cICIiAhatmzJ2LFjbel//vknwcHBuLi48Oeff3Lw4EE6dKg68evi4sJll13GF198wdixY1m0aBGjR4+u8z44msLsHACa+Dd1siWOJyU/hec3Ps+vx36lb9u+vDDgBQJ9ql08qblAGuZMyiVE69atWbBgAePGjaNnz54MGDCA/fv3k5OTw/Dhw+nZsydDhgzhjTfeAODWW2/l5ZdfJjo6msOHD9e6vaCgIGbMmEF8fDzR0dH069eP4cOHExwczGOPPUafPn24+uqriYiIwM/PD4DrrruO9evX267xwQcfcOeddzJgwABMJpOt3P3338/7779Pv379OHLkSJXIoK6IjY0lPDycyMhIJk2axMCBA215qamp+Pn50bLlmee4rF+/nqioKKKjoxk7dizz5s2zDbldc801pKWlAfDaa6/x6quv0rlzZ/Lz87njjjvqvA+OpqysFIULnk19zl24kWIRCx//8TFjvhrDrpO7eK7/c7x31XvaUTgQ5Yhff84gLi5OTj/8aN++fXTv3t1JFjU+8vPz8fHxoaysjNGjR3PffffZ5hBGjRrFm2++SWhoqK0cwMyZM8nMzGT27NnONB0wbvStWrViwoTzVohp9JSXlJKefAQ3Vy+adwhytjl1zr59+/AN8uX5jc+z5fgW+rftz/QB02nn0/iGCxsKSqnfRCTuXOX0MJTGxrPPPsuPP/5IcXExw4YNqzKp/uqrr5KamkpoaCgrV65k1qxZlJeXExISwsKFC51ndCWaN2/O7bff7mwznEpBRhYAnj4X33yFiFBQVsCElRMwKRPT+09nTNgYPflcT+jIQqO5iEg7dASLlNO6Y0eU6eIZZS41l5Kan0ri/kQ+zf6U5/s/T1ufts4266JARxYazSWGpdyMRcpwMblfNI5CRMgsziSt0JhT8vfwZ+7QuTqacAIXx3+URqOxDkEJnl5ezjalTig1l3I49zDHC47TxK0Jnf0708StiXYUTkJHFhrNRUJJUSGgaNK8ce/arogmThSeQKFo59MOfw9/7SScjHYWGs1FgCEcWIZJueHi1ni/1iXmElLzUyksK8TH3Yd23u30RroGgh6Gqif69u1LdHQ0wcHBtGzZkujo6PPaK/H555/zxx9/1Lr9QYMG2bVpryZef/11Pvroo/OuXx/cdNNNHDp0qNq8tLQ04uPj8fb25uGHH67xGtu3b7d9Vr179+b0RRMNlaKsHBqzcKCIkFGUQWJ2IsXlxQT6BBLsG6wdRQOi8f4EaWRoiXLHczaJ8gqRxoSEBNtu9OqYNm0aL730EldddRUrV67kiSeesGlfNWSK8vKBxikcWFJeQmqBEU34uvvS1rutdhINEB1ZNAC+/fZb+vfvT2xsLOPGjaOgoAAwzlYIDw8nKiqKxx9/nJ9++olVq1bxyCOPnPcObjAkynv06EFkZCRPPfWULX3evHl06dLFJvVd8Qu8OonyqKgoBgwYwGOPPWbTV0pMTOSyyy4jJiaGXr162Rzk2rVrbeqwYWFhPPPMMyxevJjevXsTFRVl68ftt9/O1KlTueKKK+jUqRMbNmxgwoQJdOvWjbvvvttm5+TJk4mLiyMiIoIZM2bY0uPj4/nuu+8wm81n9LlCTt3T8+y6/0opcnMNdeKcnJxGow3VGIUDRYT0onQSc05FE+1922tH0UC5dCKLb5+A43V8wlabHnDtKxd0CS1RXn8S5fbw9ttvc8011/Dwww8jIlUUcRsqjVE4sKS8hJSCFIrKinQ00UjQkYWTqSxRHh0dzb///W8OHz5cRaL8iy++wNu7bm4ElSXK3dzcbBLlFekBAQG4u7tXEeE7duyYTW+pOonyCkpKSrj77ruJjIzklltuYe/evba8ColyT0/PMyTKK0dI1UmUm0wmm0Q5wLJly4iNjSU2NpZ9+/ZVaadCovx8mTNnDnPmzCEpKYlZs2Y1ioOPGpNwYOVoorS8lEBfHU00Fi6dyOICIwBHoSXK61+i/GwsXbqUd999F4Bx48YxZcoUu+o5k7KyUpRq+MKBZ0QTPm1xM2kn0VjQkYWTGTBgAOvXr7et4ikoKODAgQPk5eWRm5vLiBEj+Mc//kFCQgJwbonyHTt21OgowJAoX7duHRkZGZSXl7N8+XIuv/xy+vbty7p168jOzqasrIzPP//cVqcmiXLgDInytm3b1rtEeWUqJMrHjh1rez/sdRRgqAD//PPPgDHX0rVr1zq1v64pLy7BImW4NuBT2apEE+ZSgnyDjGhCO4pGxaUTWTRQKkuUl5aWAvDyyy/j5eXFmDFjKCkpwWKxVJEov+eee5g9e/Z5ncFdWaJcRBg5ciTDhw8HsEmUBwYGniFRXnmCuUKi3NfXl8GDB1eRKB87dizLli1j6NChDpcoDw0NtVuivKLvhYWFlJWV8dlnn/HDDz/QtWtX7rzzTh566CGio6NZsGABDz74IGazGS8vL+bNm1fnfahLCjKzgYYrHFhcXkxqfipF5TqaaOxoIUGNDS1R3vhoqMKBFdHEyaKTmJSJtt5taere9IJ3YevvdN1jr5CgQ/+7lFLDlFL/U0odVEo9UU1+B6XUD0qp35VSPyqlgirlmZVSO6yPlY60U2Pw7LPPEhMTQ1RUFF27dq1Wohxg5cqVREdHExkZyaZNm3jyySedZXIVLjWJ8lPCgW4NylEUlxdzKOcQaYVp+Lr70sm/E34eflquo5HjsMhCKeUC7AeuApKBrcCtIrK3UplPgW9EZJFSaghwp4j8xZqXLyJ2z9jpyEJzqZF3Ip2C/Cy8vf3xbVP90Ft9Ul004efhV6dt6O903dMQJMr7AAdF5JDVoOXAaGBvpTLhwCPW5+uALx1oj0ZzUVFcIRzYwvm7tovLi0nJT6G4vJimHk1p690WV5OeEr2YcGTsGggkVXqdbE2rzE7gRuvzGwBfpVRz62tPpdQ2pdRmpdT57UDTaC5SRARzhXCgq/NuyhaxcLLwJIdyDlFmKbOtdNKO4uLDkZ9odQOUp495TQP+qZS6A9gApAAVi+6DRSRVKRUK/FcptUtEEqs0oNRkYDJAcHBwXdqu0TRoKoQDHbHizF50NHFp4chPNhloX+l1EFBla62IpAJjAJRSPsCNIpJTKQ8ROaSU+hGIARJPqz8fmA/GnIVDeqHRNEAqhAOdcXaFRSykF6WTXpSOSZlo79ueph4Nf/e45sJw5DDUViBMKdVRKeUO3AJUWdWklGqhlKqw4UngA2t6gFLKo6IMMJCqcx2NDi1R7njOJlEO8NJLL9G5c2e6detWo5KsiPDEE0/QpUsXunfvzpw5c2zpU6ZMoXPnzvTs2bPKe7lgwQLCwsIICwtj6dKltvStW7cSGRlJ586deeSRR2zpGRkZXHnllYSFhXHNNdeQk5NT6zYM4UA3duz63WFtVNePBx56gD9z/uRk4UnMeWam3DSFXpG9LriNJ554gqCgIPz9/Wv8/DRORkQc9gCuw1gRlQg8bU2bAYyyPh8LHLCWeR/wsKYPAHZhzGnsAu4+V1u9evWS09m7d+8Zac7mww8/lKlTp553/dtuu02++OKLWtcbOHCgJCQknFebpaWlEhUVJeXl5edVv75Yu3at3HvvvdXm7dy5U2JiYqSkpEQOHjwonTt3FrPZfEa5+fPny5133ikWi0VERE6cOCEiIl999ZWMGDFCRER++uknGTBggIiInDx5UkJDQyUrK0vS09MlJCREsrOzRUQkNjZWtmzZIhaLRa666ir5/vvvRUTkkUcekddee01ERF588UV56qmnatVGhw4d5H8Jv0nG0RSHtXF6Pzb/ulmO5x+XAfEDZMFnCySnOKdO29i4caMkJSWJn5/fWT/jhvidbuwA28Se+7k9hRrDozE7i1WrVkm/fv0kJiZGbr75ZsnPzxcRkWnTpkn37t2lR48e8re//U02bNggAQEBEhISIj179pQ///zT7nYrO4slS5ZIZGSkREREyJNPPmkr869//UvCwsLk8ssvl7vvvlseeughERH5z3/+I3fffbet3KZNm6RHjx7Sv39/mTZtmvTs2VNERA4ePCiDBg2S6Oho4wazebOIiKxZs0bi4+PlxhtvlM6dO8vTTz8tixYtkri4OOnRo4etH7fddptMmTJF4uPjJTQ0VNavXy9//etfpWvXrnLXXXfZ2p80aZL06tVLwsPD5YUXXrCll5eXS0hISLVObcaMGTJr1izb6yFDhsiWLVvOKBcTE1Pt+3rXXXfJJ598YnsdGhoqaWlpsnjxYpkyZcoZ5Y4ePSrh4eG29MrlKuqKSJVy9rZx27hbZP7bb8mBvfsc1kblfnQP7y4Hsw7K7pO75c15b8q9991bp21UUFZWpp2FE7DXWVwys1GvbnmVPzJrP3xzNro168bjfR6/oGtoifL6kShPSUkhPj7e9jooKIiUlBR69+5dpdyff/7J0qVL+fLLL2nVqhXvvPMOnTp1IiUlhfbt259Rv7bpYAwRVUiSBAYGcuzYMZuN9lyrdatWHE87SXpOtsPaCAoKIjk5mV2Ju2jWuhnllnLa+7YnKiyKdavW1VkbFfZqGj4NZ9vnJYqWKK8fiXKpZvNpdTuKi4uL8fX1Zdu2bdxxxx1MnDjxrPVrm14dFen2XKu8uATEjIvJ1WFtAJRZysgoySCrKAtXkyud/DvZJrHrqo2zXUvT8LhkIosLjQAchYiWKK8PifKgoCCSkk5t+0lOTq72FLzAwEBuvNHY+nPjjTdyzz33ANjq9+vXr0r9oKAgNm/eXOW6kZGRZ22vefPmnDx5kpYtW5KSkkKbNm3sbqMgI5tjx08QExPnkDYsYuFk0Un+OPQHlw+9nNgusZw8dtK2JLau+lH5vdI0DnRk4WS0RHntOF+J8lGjRrFs2TJKS0tJTEzkyJEjVYbWKrj++uv573//C8C6detsZ52PGjWKxYsXA/Dzzz/TunVrWrZsybBhw/j222/Jzs4mIyODH374gauvvpr27dvj4eHB1q1bERGWLFnC6NGjbddatGgRAIsWLaqSfq42jh0/xk+bNjHihtF13saqb1ex4+gODiYfZOvPW7n9+tvp3qm7Q/pR+b3SNBLsmdhoDI/GPMH9/fffS1xcnERFRUlUVJR88803kpSUJL1795aoqCiJjIyUxYsXi4jI+vXrpVu3bhc0wb148WLbBPcTTzxhK/Puu+9KWFiYxMfHy+TJk+W5554TEZHExESJj4+3lfvll18kMjJS+vfvL48//rgMHjxYRET++OMPiYyMlL59+8rTTz8t3t7eImJMcI8ePbpaWyrnVV7pdeDAAdvEeeU8i8Uit99+u3Tv3l2GDx8uo0ePliVLloiISEpKivTr16/G9+CFF16Q0NBQ6dKli6xevdqWfvXVV9tWPWVkZMiwYcNs/fv9999FRMRsNss999wjoaGh0qNHD9m+fbut/vz586VTp07SqVMnWbRokS198+bNEh4eLqGhofLggw/aVlilpaVJfHy8dO7cWa666irJzMy0u42Q4GB55/U36rQNs8Usx/OPy/Q3pktwx2DpGNrR4f04/b165JFHJDAwUJRSEhgYKC+++GK1n2FD/E43drBzgltLlGtsaInyho1NONAnAN/WLerkmkVlRaTkp1BiLsHfw5823m1wMbnUybUdgf5O1z0NQUhQ08h49tln+fHHHykuLmbYsGHVSpSHhoaycuVKZs2aRXl5OSEhISxcuNB5RlfiYpcoLy60Cgc2v/CNaxWaTulF6biaXAluGoyvu++FG6m5aNGRhUbTCBCLhRN//olJudIqtMMFXauwrJDU/FQjmvD0p02Thh1NVEZ/p+seHVloNBcRRVm5XKhwoEUspBWmkVGUoaMJTa3RzkKjaQQU5V+YcGBhWSEp+SmUmksJ8AygdZPWjSaa0DQMtLPQaBoBFcKBbp61iywqRxNuJjc6NO2Aj7vdB1BqNDa0s9BoGjgleQWImHF396xVveLyYpLyknQ0oakT9Ka8ekJLlDue0yXKa5IIr8zatWvx8/OzfR4zZ8605b3xxhtEREQQERHBO++8Y0tPSEigX79+9OjRg9GjR5NvHSIqKSlhwoQJ9OjRg+joaDZs2GCr89FHH9GjRw8iIiJ48sknbemHDx9myJAhREVFccUVV1SRKpk2bRqRkZFEx/Xim2+/w9vfz2ZzbGwskZGR3HXXXbYd8JmZmYwaNYqoqCh69+nNmi1rsIiFDk07sHz+cqJ6RDm9L5GRkXz22We2dJHqJeE1DRB7NmM0hkdj3pRXG7REec2cLlFek3x3ZU7fMFhBQkKCREVFSWFhoZSWlkp8fLwcOnRIRESio6Pl559/FhGRefPmyfTp00VE5M0335SJEyeKiMixY8ekV69eYrFY5MSJExIcHCzp6elisVhk/Pjx8uOPP4qIyPXXXy9Lly4VEZHVq1fLHXfcISIiX375pVxzzTVSXl4uib/vlqiICMnLy5Py8nIJDAyUgwcPiojIk08+KQsXLhQRkYcfflheeuklyS/Nl69+/koGXD5ASspLGlRf8vLyJDY2VvLy8kSkZkn4mmiI3+nGDnZuytORRQPg22+/pX///sTGxjJu3DgKCgoAeOyxxwgPDycqKorHH3+cn376iVWrVvHII4+cV1RSwdKlS+nRoweRkZE89dRTtvR58+bRpUsX4uPjmThxIg8//DAAa9asoXfv3ri4GEMYmzdvJioqigEDBvDYY48RHR0NQGJiIpdddhkxMTH06tXLpji7du1amzpsWFgYzzzzDIsXL6Z3795ERUXZ+nH77bczdepUrrjiCjp16sSGDRuYMGEC3bp14+6777bZOXnyZOLi4oiIiGDGjBm29Pj4eL777jvMZjNJSUkUFxfTu3dvlFL85S9/4csvv7T7Pdq3bx/9+/fHy8sLNzc3Bg8ezBdffGHr58CBAwG46qqrWLFiBQB79+7lyiuvBKBNmzZ4e3uTkJBAYmIi3bt3p3nz5iilGDp0aLV1rrzySpvMyt69e4mPj0fKymnSxJ3uXbvz/fffk5aWho+PD506daq2/f6D+3Mk9wjdw7uT/Gcyedl5DaYvLi4u+Pj4EBkZadM5mzt3Ls8995xNULBVq1Z2f0aa+uWSmbM4/vLLlOyrW4lyj+7daFPpZns+aIlyx0iUFxUV2S2H/fPPP9OzZ08CAwN5/fXXCQ8Pp0ePHrzwwgtkZmbi4eHBt99+a7upduvWjf/85z8MHz6cTz/91Cbm17NnT7788ktuuukmDh8+TEJCAklJSQwcOJDdu3dz9OhR2rZty1dffWW7Ofbs2ZMVK1YwdepUVqxYQW5uLjk5ObbP446x40jPzGTT1l/pP3gQ119/PYWFhSQkJBAdHc2KFSts7XeL7MbSj5fy9EtPk7onleTkZJKTkxtMXx566CHy8/NZv349sbGxQM2S8JqGh44snIyWKHeMRLlUs9m0Ojns3r17c/jwYXbu3Ml9993HmDFjAIiMjOTRRx9l6NChXHvttcTExODqavy2WrhwIW+++Sa9evWiuLgYNzc3ACZNmkTr1q3p1asX06ZNo3///ri6utKiRQvmzJnD2LFjufzyy+nYsaPtWv/4xz9scxCbNm2iTZs2uLq6ct111zF06FCGjhrO1Ef/jwEDBuDq6orJZOKjjz7igQceoG/fvjRt2hRXV1dySnIYP3U82RnZjIkfw/vz36dnz564urqSd6C0AAAgAElEQVQ2mL7079+f2267zXYtqFkSXtPwuGQiiwuNAByFiJYod4REub2S5H5+frbnI0eO5L777rNFcZMnT2by5MkA/O1vf6Nz584AhIeHs2bNGsAYYvnuu+8AcHNz46233rJdr0+fPoSFhQEwevRomyrru+++i5eXF2BIolcMCeXm5rJixQrbD4NnnnqaybeNx8XkztTHp9muNWjQIH7++WcAVq1axf7E/STnJdOqeSs+/fenuJhcsFgshISEEBISAuD0vjz33HO2/8mbb77Zdq2aJOE1DQ8dWTgZLVFeO+yVKD+bfHdljh8/bnu+efNmXF1d8fc3tJfS0tIAY5XPV199xbhx46qkWywWXnrpJe69917A+OwKCwsBYx7Kx8eHLl26VKmTmZnJv/71L9scTHp6uu19evnll22/rMvLy0k6eAgQ/pd4kH379tnmAyquVVxczMuvvMzI20bi7eZNU3NTLGYLYMw/DR061HazdnZfMjMzAWP1VeW+1CQJr2l4ODSyUEoNA94CXID3ReSV0/I7AB8ALYFM4HYRSbbmTQCesRZ9SUQWOdJWZ9G6dWsWLFjAuHHjKC0tBYwvmpeXF2PGjKGkpASLxcIbb7wBwK233so999zD7Nmz+fLLL22/HO0lKCiIGTNmGJOnIowcOZLhw4cDxoR6nz59CAwMJCIiwvar+7rrrqsywfzBBx9w55134uvry+DBg23l7r//fsaOHcuyZcsYOnToBUlT1ERsbCzh4eFERkYSGhpqG3sHSE1Nxc/PzzZkNnfuXO644w6Ki4sZMWIEV111FQBz5szBw8ODiRMnsnz5ct577z3c3Nzw8vLi448/tl3v+uuvJzs7G3d3d+bNm2fr55IlS5g/fz4iwk033cRf/vIXwHA81113HSaTiaCgINtZDwBTp05lz549AEyfPt02Lv/DDz/wzDPPoJQiPj7eNmFfUlLC1SOvQ0Ro3qIF//73v20LDP7+97/z3XffUWYu4+Y7b+aK+CsI8g3il59/4a677sLFxYXIyEjef//9BtOXQYMGAUYkV7kvTz31FLfddhuvvfYavr6+zJ8/v/b/FJr6wZ4lU+fzwHAQiUAo4A7sBMJPK/MpMMH6fAiwxPq8GXDI+jfA+jzgbO01lqWzDZmK5YylpaVy7bXXysqVK215I0eOlMTExCrlREReeuklefTRR+vX0BqYNWuWbRlpY8diNsuxgwflROLhM/MsFkkrSJPdJ3fL0dyjYraYnWChc9Df6bqHBrB0tg9wUEQOiUgpsBw4fRwgHPjB+nxdpfxrgDUikikiWcAaYJgDbdVgSJTHxMQQFRVF165dq5UoB1i5ciXR0dFERkayadOmKhuznMnFJFFek3CgiJBWmEZaYRp+Hn4E+QRhUno0WeN4HDkMFQgkVXqdDPQ9rcxO4EaMoaobAF+lVPMa6gY6zlQNGKtZaqKyLPT48eOrrIJqKNx1113ONqHOqBAO9K4kHCgiHC88TmZRJgGeAbT1blvtCi+NxhE48idJdf/Fp894TgMuV0olAJcDKUC5nXVRSk1WSm1TSm07efLkhdqr0TQYKoQDXa3CgSLCsYJjZBZl0syrmXYUmnrHkc4iGWhf6XUQkFq5gIikisgYEYkBnram5dhT11p2vojEiUhcxaSmRtPYsQkHurkDhqNIyU8hqziLFl4taNOkjXYUmnrHkc5iKxCmlOqolHIHbgFWVi6glGqhlG3A9UmMlVEAq4GrlVIBSqkA4GprmkZz0VOQnQNAE38/LGIhOT+ZnJIcWjVpRWvv1tpRaJyCw5yFiJQD92Pc5PcBn4jIHqXUDKXUKGuxeOB/Sqn9QGtgprVuJvAihsPZCsywpmk0Fz1lZaUo5YKbjxdJeUnkluTSxrsNLZvo6FnjPBy6jEJEVolIFxHpJCIVjuA5EVlpff6ZiIRZy0wUkZJKdT8Qkc7Wx4eOtLM+0BLljud0ifInnniCoKAg2ya76li8eLHts4iOjkYpxe7du4G6leK2R1a8b9++7EzYgUgZri7uPPfyc1zZ50puHHwjH71/6r13tqy4vX2pLMNSk9z79u3bbd+N3r172zZ7ahog9qyvbQyPxrLPQkuUO47TJco3btwoSUlJ4ufnZ1f97du3S1hYmIhInUpx2yMrLiKye/duib9ssBw7uF9Wr/6PdInoIqmZqQ1KVrw2fRk6dKiInF3u/YorrrBJx3/11Vdy5ZVXnvUzaojf6cYODWCfhcZOtES50Y+6lCgH6N+/P23atLH7fVm2bBm33nqrrS91JcV9LlnximtFRERwMDGRzKwcdh7ew8D+A2kb0LZByYrXpi/79+8nIyPjrBLpSilyc3MBQy6mOv0uTcPgkhES/OmT/aQn5dfpNVu09+Gym7tc0DW0RLljJMp79uxZq/dFRPjkk09sQnphYWF1JsV9Nlnximv169ePDRvWk3r8GMdPpDG492AWvLGgwcmK29uXTZs22SWR/vbbb3PNNdfw8MMPIyJs2rSpVp+bpv7QkYWT0RLljpEory0bN24kICDAJmRXl1LcNcmKAzz99NOkpaXRM7on77z9FuHduuHp7U3fmL4NUlbcnr5ER0fzr3/9yy6J9Dlz5jBnzhySkpKYNWsWkyZNqvVnp6kfLpnI4kIjAEchWqLcIRLltWX58uW2IagK6lKK+3RZ8QrH5+fnx3sfvMfh3MP4ZrgSN3gQPWJigIYrK362vlQIDtorkb506VLeffddAMaNG8eUKVPO9jFpnIiOLJyMliivHfZKlNcGs9nMZ599xi233FIlvS6luCvLis+aNcsmBX4i/QT70/djsVhY/NFSBg8chK9f0yp1GpqseE19qfjfAfsl0lu3bm1zPGvXrqVr1652fGIaZ3DJRBYNFS1RXjtqI1H+6KOP8sknn5Cbm0tQUBD33nsvzzzzDF988QW7du2yOdV169bRqVMngoODq7RVl1LcFbLiFouF+++/n8GDB1NcXszaX9fy1INP4WFyp0unTsx5421b+w1VVry6vgDs2rWr1hLpCxYs4MEHH8RsNuPl5cW8efNq8d+gqU+UI379OYO4uDg5fY32vn37qgjgac5Ofn4+Pj4+lJWVMXr0aO677z7bHMKoUaN48803CQ0NtZUDmDlzJpmZmcyePduZpgPw2muv0apVKyZMmOBsU85JUXkRR3KPoFCENA0hP+UkZeVFtAgMtulBac5Ef6frHqXUbyISd65yehhKY0NLlNcPhWWFHM45jAkTHf064uHqcYZwoEbT0NCRhUZTjxSUFnA07yiuJlc6NO2Au4s7JXn5ZKUdw8Pdm4D2ep/B2dDf6brH3shCz1loNPVEXmkeSXlJuJvc6dC0A24uxpLXgmxjU1oTfz9nmqfRnBXtLDSaeiC3JJfk/GQ8XDzo0LQDrqZTX70K4UAP37rZS6PROAI9Z6HROJickhyS8pLwdPE801EUl9iEAzWahoyOLDQaB5JVnEVqfipN3JoQ7BuMi8mlSn5hRhYAXr4+1VXXaBoMOrKoJ7REueM5XaL8qquuIjo6moiICKZMmWITGKyMiDBlyhQ6d+5Mz549L+g9Op2MogxS81PxcfehQ9MOZzgKMPYzgAmvAD1foWnYaGdRT/z666/s2LGDGTNmMG7cONtu69puqjtfZ3EhlJWVsWTJEtuu24bKvffey2uvvWZ7vWLFCnbs2MGuXbtITU21yVpU5uuvvyYpKYmDBw8yZ84cpk6dWie2pBelc7zgOL7uvrT3bY9JnflVM5eXY5EyXFzc9Ol3mgaPdhYNAC1RbvSjriXKmzY1ZDPMZjMlJSXV3pC/+uor/vrXvwJG9HX8+HFOnjx5Xu8rGJFKWmEaJwpO0NSjKUG+QdU6CoDC9CxA8PRqct7taTT1xSUzZ7Fu4XzSjhw6d8Fa0KpDKFfcMfmCrqElyh0rUT506FB+++03RowYwQ033HDG+5GSkkL79u1tr4OCgkhJSbFJhtQGEeFE4QkyijLw9/CnnU+7s0YMxUVFgMK7eUCt29Jo6hu7Igul1Aql1HClaviJVHO9YUqp/ymlDiqlnqgmP1gptU4plaCU+l0pdZ01PUQpVaSU2mF9/Ks27TYmtES5YyXK165dS2pqKnl5eaxfv/6M96O6TannMyQkIhwvOE5GUQYBngHndBRisWC2lGFSbphcz5zL0GgaGvZGFnOBO4G3lVKfAgtF5KwD50opF2AOcBWQDGxVSq0Ukb2Vij0DfCIic5VS4cAqIMSalygi0fZ35excaATgKLREueMlyr28vBg5ciRfffUVV1xxRZW8oKAgkpKS6NevH2BEXrU9rU1ESC1IJbs4m+ZezWndpPU5HU5hVg5gwcPTs1ZtaTTOwq5IQUTWishtQCxwGFijlNqolLpTKeVWQ7U+wEEROSQipcByYPTplwaaWp/7AbU/taaRoyXKa4e9EuV5eXkcP34cMOS2v/32W9vBRpUZNWoUixcvBuDnn3+mdevWtRqCsoiFlPwUsouzadmkpV2OAqA435iX8m7mb3dbGo0zsXvOQinVHLgd+AuQAPwbGARMAOKrqRIIJFV6nQz0Pa3MdOB7pdQDgDcwtFJeR6VUApALPCMiP9lra2NCS5TXDnslylNTUxk9erTt/Rs6dKjtFLY5c+bg4eHBxIkTGTlyJN9++y2dOnXC29u7iqz3ubCIheS8ZPJK82jt3ZoWXi3srltuLsWkhQM1jQkROecD+BzYCzwJtD0tb1sNdW4C3q/0+i/AO6eVeRT4P+vz/tY2TIAH0Nya3gvD6TStpo3JwDZgW3BwsJzO3r17z0jT1ExeXp6IiJSWlsq1114rK1eutOWNHDlSEhMTq5QTEXnppZfk0UcfrV9Da2DWrFmycOHCemnLbDHL4ZzDsvvkbkkvTK9V3aKcPDl2cL9kHk1xkHUXL/o7XffUdA8//WHvhPU/RSRcRP4uIsdOczY1qRUmA+0rvQ7izGGmu4FPrNfZBHgCLUSkREQyrOm/AYnAGeeiish8EYkTkbjzWb2iqYqWKLcPs8XM0dyj5Jfm086nHc29mteqfqEWDtQ0QuwdhuqulNouItkASqkA4FYRefcsdbYCYUqpjkAKcAsw/rQyR4ErgYVKqe4YzuKkUqolkCkiZqVUKBAG1O26V80Z/OMf/6gxr7Is9Pjx46usgmoo3HXXXQ5vw2wxcyTvCEVlRQT6BuLvUfs5h7LyEi0cqGl02BtZTKpwFAAikgVMOlsFESkH7gdWA/swVj3tUUrNUEqNshb7P2CSUmonsAy4wxoWDQZ+t6Z/BtwrIpm16VglO86nmkZzBuWWcg7nHqa4vJj2vu3Pz1EUlSBSroUDzwP9XXYu9kYWJqWUst7IK5bFnvO/XURWYSyHrZz2XKXne4GB1dRbAayw07Ya8fT0JCMjg+bNm2s5Bc0FUWYu40jeEUrNpbT3bY+vu+95XacgUwsHng8iQkZGBp56qbHTsNdZrAY+sW6OE+Be4DuHWVVHBAUFkZycfEHyDRqN2WImozgDs5hp5tmM5LTk875WXnoGIhZ8S8pQJ46du4LGhqenJ0FBQc4245LFXmfxOHAPcB+ggO+B9x1lVF3h5uZGx44dnW2GphGTlJfExNUTyS3NZe7QuUS3Ov99ooXpWcyd/iQB3iHc9cGbdWilRuN47HIWImLB2MU917HmaDQNh0M5h5i0ehIllhLev+Z9IppHXND1tixaAZQTFh1TNwZqNPWIXc5CKRUG/B0Ix1ixBICIhDrILo3Gqfwv839MXjMZheKDaz6gS8AZK7drzcFdvwNuxP11zIUbqNHUM/auhvoQI6ooB64AFgNnihlpNBcBu9N3c9fqu3A1ufLhsA/rxFGYS8vILT6Gt3sbvPzPb3Jco3Em9joLLxH5AVAickREpgNDHGeWRuMctp/YzsTvJ+Lr7suiYYvo6Fc3c147P/4GkSJCOp+pT6XRNAbsneAutsqTH1BK3Y+xya6V48zSaOqfTambeGjdQ7Ru0pr3rn6PNt5t6uzae37eBJjoq4egNI0UeyOLh4EmwIMYWk23YwgIajQXBRuSN3D/D/cT6BPIh8M+rFNHAZCRk4KHS0sCOrY/d2GNpgFyzsjCugHvZhF5DMjHONdCo7loWHNkDX/b8De6BHRh3tB5+HvWrWx44tqNmCWH4LZ6CErTeDlnZCEiZqCX0lugNRchXyd+zbT104hsHsn7V79f544CYPvXxqFVcWOG1/m1NZr6wt45iwTgK+speQUViSLyec1VNJqGzaf7P+XFTS/Sp00f3h7yNk3cmjikneMnj+BiCiB4YKxDrq/R1Af2OotmQAZVV0AJxjkXGk2jY+nepby69VUuC7yMN+LfwNPVMZpDJ/84RKk5nTbNIh1yfY2mvrB3B7eep9BcNLz3+3u8nfA2Q4OHMmvwLNxcajoZ+ML5denngNDjissc1oZGUx/Yu4P7Q4xIogoi4vgDBDSaOkJEeCfhHd7b9R7XdbyOmYNm4mqy+2Th8yLp8AGU8ibixmsc2o5G42js/aZ8U+m5J3ADZ556p9E0WESE17a9xpK9S7gx7Eae7fcsLiYXh7ZZmJ5FYdkJArw74uLi2LY0lzAF6cajlWNX29k7DFXlbAml1DJgrUMs0mgcwL/3/Zsle5cwvtt4Hu/zOCZl7xaj8+fXhZ8B5XSJ0cKBmjrCXA4ndkPyVuORtAWy/oTAOJj0g0ObPt8YPAwIrktDNBpHkZyXzNsJb3NZ4GU80eeJejsIK3H3LsCd3hNuqJf2NBch+WmnnELyNkjdDmWFRp5PawjqDb3ugOD+DjfF3jmLPKrOWRzHOONCo2nQiAgzNs1AoXi237P15igqhAN93Fvj0VQLB2rswFwGx3cZTiF5i+Egso8YeSZXaBMFsX81HERQb/APhnrc/mbvMJT+b9c0SlYmrmTTsU081fcp2vq0rbd2dyz7GpEiOnTuXm9tahoZeSdOOYXkbZCaAOVFRp5vW8Mh9J4I7ftA257g5uVUc+2NLG4A/isiOdbX/kC8iHx5jnrDgLcAF+B9EXnltPxgYBHgby3zhPXcbpRSTwJ3A2bgQRFZXZuOaTTpRenM2jqLmFYxjOs6rl7b3vPLZgzhQD0EpQHKS61Rw1arg9gKOUeNPJOb4Qzi7jwVNfgF1WvUYA/2zlk8LyJfVLwQkWyl1PNAjc7Cqik1B7gKSAa2KqVWisjeSsWeAT4RkblKqXBgFRBifX4LEAG0A9YqpbpYpUc0Grt4ZcsrFJUXMX3A9HqZ0K5MZm4KHq5aOPCSJfdY1ajh2A4oLzbymgYaDqHvPUbU0CYK3ByzKbQusddZVPdNO1fdPsBBETkEoJRaDowGKjsLAZpan/txajnuaGC5iJQAfyqlDlqvt8lOezWXOP89+l9WH17NAzEPEOpXvwc6HlzzsyEc2EYPQV0SlJfAsd8N55C81YgacpONPBd3aBttDCfZooZA59p7ntjrLLYppd7AiBQEeAD47Rx1AoGkSq+Tgb6nlZkOfK+UegDwBoZWqrv5tLqN8x3W1Dt5pXnM3DyTLgFduDOy/sUHEr4xVpXHjbmu3tvW1AM5yaecQvIWOLYTzKVGnl97I1oImmqNGnqAq4dz7a0j7HUWDwDPAh9bX3+PMYR0NqobcDt9F/itwEIRma2U6g8sUUpF2lkXpdRkYDJAcLBeyasxeOO3N0gvTuftIW/jZnKclEdNHDt5BFctHHhxUFZsOIPKUUOedQDE1dOIGvreA0F9jKihaf0toqhv7F0NVQA8UctrJwOVB2yDOHPX993AMGsbm5RSnkALO+siIvOB+QBxcXFnOBPNpcfW41v5bP9nTAifQESLiHpvP23vQcrM6bRp1qPe29ZcICKQk3Ra1PA7WMqMfP9g6DDAcArte0PrHuDq7lyb6xF7V0OtAW4SkWzr6wCMOYWzCd5sBcKUUh0xjmG9BRh/WpmjwJXAQqVUdwwpkZPASuAj69BXO4xNgFvs7pXmkqS4vJjpG6cT5BPE1JipTrFhy7+/BISoIVo4sMFTVgSpO6pGDfnHjTxXL2gXA/2nnIoafFs7114nY+8wVIsKRwEgIllKqbOewS0i5dbzuldjLIv9QET2KKVmANtEZCXwf8B7SqlHMIaZ7hARAfYopT7BmAwvB6bqlVCaczF351yO5h3lvavfw8vVOWvSjx4xhAPDx1ztlPY1NSBibHBL3mZdobTFWMpqKTfyA0Kg4+BKUUMkOFCNuDFir7OwKKWCReQogFIqhGrmEE7Humdi1Wlpz1V6vhcYWEPdmcBMO+3TXOLszdjLoj2LGBM2hn5t+znFhvy0DIq0cGDDoLTQ2OSWvOWUgyhIM/LcmkC7WBjwgDVqiAOfs/721WC/s3ga+Fkptd76ejDWiWWNxtmUWcp4fuPzBHgG8GivR51mx9bFnwPldI1tGMKBvx3JZPGmI0S0a0pMcAA9Av3wdLsInZgIZB2upKG0BY7vhorBiGah0GmI4RTa94FWEeDiWGn6ixF7J7i/U0rFYTiIHcBXQJEjDdNo7GXRnkX8kfkH/4j/B34efk6zo0I4MK4B7NpOyyvmniXbyS0u46sdxtoQNxdFeFvDccQE+xMbHEBQgFe96WXVGaUFkLL9lPJq8lYoOGnkuXlDYCwMevhU1ODdwrn2XiTYO8E9EXgIY1XSDqAfxga5IWerp9E4msM5h5m7Yy5Dg4cytMPQc1dwEGUlJVbhwDZOFw40W4SHl+8gv6SMr+8fRDNvdxKOZrH9aDYJR7P4eGsSCzceBqClrwcx7f2J7RBArDX68HJvQNGHCGQeqhQ1bIUTe05FDc07Q+erKkUN4eDgc0ouVeyNxR4CegObReQKpVQ34AXHmaXRnBuLWJi+aToerh481fcpp9qyc9l/ECkiJMyxB9DYw9s/HGBjYgazxkbRtY3huK6OaMPVEW0AKDdb+ON4ns2BbD+axfd7TwDgalJ0b9uU2GB/YoINB9K+WT1GHyX5kPJb1aihMMPIc/eBwF5w2aOnooYmzerHLo3dzqJYRIqVUiilPETkD6VUV4daptGcg8/2f8ZvJ37jhQEv0LJJy3NXyDoMXgHgWfdDVXs3bgJM9JlwY51fuzb8fCCdt/97gBtjg7g5rnpdKlcXE5GBfkQG+vEX6zEIGfklJFgdx/ajWXz6WzKLNhny2C183KsMXUUF+dHEvQ7G/EUgI7Hq0tW0PSAWI79FF+gyzLpCqQ+07KajBidi7yeebFWa/RJYo5TKQh+rqnEixwuO88Zvb9C3TV9u6GzHHMF/Z8KGWcZz75bG8EXzTsbfZhV/O563DHRGhXBgB+ep0qTlFvPwxwl0bunDi9fXbkNicx8Phoa3Zmi4sZeg3GzhfyfybENXCUezWWONPlxMim5tfIkNDiC2gz8x7QPo0LzJuaOP4lzj8J6KDW/JW6Eoy8jzaGpEDYMfM5xDYC8dNTQw7J3grvg2TldKrcMQ/fvOYVZpNGdBRJi5eSZmi5nnBzx/7pvUvq8NRxFxgyHPkJlo/KI9sAYSllYqqAxtn+ahVmfS+ZRT8QuucQXN/tU/YZFcAtvW/47xCsrNFh5YlkBBiZllk2Iv+Je/q4uJiHZ+RLTz4y/9OgCQWVDKjqQsth8xIpDPtyezZLMRfTT3difGOnQVE+xPz8CmeOcdPi1q2IttxX2LrtBtuDGc1L6PEUXoqKFBU+v/KBFZf+5SGo3jWH14NT8m/8i0uGm09z2HBPjJ/8EX9xpnFN8w70xRt+LcU84jIxEyDhqP3z+FkpxT5UxuxsatyhGJ9e+OVVbhwBudJxz41g8H+PXPTGbf1JOw1o6ZYG/m7c6Qbq0Z0s2IPswWYf+JPLYfzWLvoWRKj/5Cyf5dlJoOUGY6CKoAgFJXX8rb9cLr8r+h2vcxPgsvf4fYqHEcerGxplGRXZzN37f8nYjmEdzW/bazFy7OgeXjjU1Y45ZUr/7p2dSQdWh32t4IEWNitcJ52B6H4NC6U2cTAMdPjMZV+dM++S34oVPViKQehlI27D/JP9cd5Oa4IG7sFeTw9rBYIH0/Lslb6J68le5JW+HkH4AgbooCv8786TGUT8o68U1mELvyWyH7TQQkuxET7Eds8Eligsvp2d4fHw99C2os6E9K06h4bdtr5JbkMv+q+biazvLva7HA55ONSe0JX0PTdrVrSCljfb53Cwg+bUe4xQK5KZBxkLTtCZTt+y9tPdrCsW2wd+WpZZ1gTKhXdh62OZJO4O5dO5uq4XhOMQ9/vIMurXx5YVTkBV+vWoqyIWWbda5hq/G82Bp1efoZcwwRN0D73qjAXvh4+tED6AHcbREOpuUbE+dHskhIyua/fxg7qU0KurT2ta66Mpbvhrbwbnz7Pi4RtLPQNBp+SfmFlYkrmdRjEl2bnWMx3vpXYf93cN3rhlJoXWIygX978G/Plg+2ARA1+ka48T3j+Mzso2dGJH9ugJ3Lql7Ht53VgXSqOkfi38EuNdNys4UHlyVQXGZmzm2xdbM/wmIxooTKx3+m/8+aqYx9DBE3WA/y6WPYa6r5FEIXk6JrG1+6tvHl1j7GMQI5hWUkJJ3a9/HN76ks22IcMern5WZbdRUbHEDP9n74emqNpoaAdhaaRkFhWSEvbHqBjn4duafnPWcv/McqWP8KRN9mnFDmQI4eOYhS3nS//iojwdUdWnQ2HqdTWmBsMKs8pJVx0IhGijJPlVMuhhz26RFJ807QNMh2c35jzX62HM7kzXHRdG7lc34dKMqqJK631djjUJJr5HkFGE6hx02GuF67WGPY7gLxa+JGfNdWxHc19JgsFuHgyXxj38eRbBKSsli//yQiRoDXpZWvbdVVbAd/Qlv4YDLp6KO+0c5C0yh4O+FtjhccZ9G1i/BwOcvJYyf3G8NP7WJg+BsOPfS+Qjiwmb3Cge7exslpbao566Iw8zRHYn0c2QhlBafKuXpCs1BOuv3v3hQAABz/SURBVAfhc9iT1zuFc32zZpDvbgyZna2/FrMRNVQ4huStkL7fyFMmQzMp8kbrSW99DOdUD0NCJpOiS2tfurT2ZVxva/RRVMbOJGPVVcLRbP7z+zGWbTEO3mzq6Up0xdBVcADRwf401dGHw9HOQtPg2ZG2g4/2fcS4ruOIaXUWkb7iXPj4NmMie9xScPN0qF1bF1mFA3vVwYl4TZoZj6C4qukikHf8lPPITKT4+H7yDu1mkusJ3FJWwoevGGU9/M5c9uvWxLq3YYuhp1SaZ5T1amY4hahx1n0NseDhXJmSyvh5uTG4S0sGdzE2W1oswqH0fNvQ1fYj2bz1wwFb9NG5pY9t30e3Nk1p5u2OfxM3fDxc9RxIHaGM4yMaP3FxcbJt2zZnm6GpY0rNpdz89c0UlBfw5egv8XarYVLYYoFP/gL/+xb++hV0dPzhQ+/f8QA5RSncv2AJHj4XPlltD2VmC7fO38y+Y7msnNqPTm6Zp4azbI9E48S3ij0NygVaR5zaCR3U21BibeQ30bziMnYm5VijD2PyPLuwrEoZNxeFn5c7AU3cCGhiOJCKv/5NjPSKvwFWB+Pv5Y67a83zMBcbSqnfRCTuXOV0ZKFp0Ly/630ScxKZc+Wcmh0FwE+z4Y9vYNgr9eIobMKBHm3qzVEAvP79/9h2JIu3bommU2t/wN+48YedJqJYVgxZfxqrllpHgsd5zmk0YHw93RgU1oJBYYaqrIhwKL2AQycLyCosJbuwlKzCMuNvQRlZhaUcyShkh9WplJotNV7bx8MVPy83ArwrnMtpjqUax9PU8+KOYrSz0DRYDmQd4L1d7zE8dDiDgwbXXHD/alg30xhS6Xtvvdi2c9nXiBTTMax7vbQH8N8/TjBv/SHG9w1mdPQ5ZEXcPKFV/dnWEFBK0amlD51antsxighFZWayCsvIKiglu7CsioPJKiwlx/o3q7CMpMxCsgrLyC0uo6bBGBeTwt/LrZITORWx+HkZaTaH433K0Xi4No6d69pZaBokZouZ6Run4+vmy+O9H6+5YEYirJhkTBqPeLPehlb2bvwVMNH7r2Pqpb2U7CIe/WQn4W2b8tyI8Hpp82JGKUUTd1eauLsS6G+/HpjZIuQUnXIshpOxRi+nRTLJWYXsTjHKlpTXHMU0cXexOhnDiVSNXqpGMhWvm3q61fuKMO0sNA2Sj/74iN/Tf+eVy14hwDOg+kIlecYObZOLMaHt3qTe7DOEA1vVi3BgmdnCAx9tp9wszLkt9uI87a6R4GJSNPN2p5n3uffBVKa4zGw4k4KyKtFLdZHMsexc43VRGZYaohiTwhat+DdxIyrIn+mjHKtN5lBnoZQaBrwFuPx/e3ceHUWdLXD8exOWsBnCjmwGDRFFFomgog6K4DpugyyOPpTnU5/iyLgMoqNBRkdH36APt1HRAXwsKqhEQETHBRm2sARCEhCIkEQIq0BYQrb7/uhCG0jokHSlupP7OacP3dW/qtw6pHO76lbdHzBBVV847v2Xgcudl/WBFqra2HmvGEh13stS1RvcjNWEjpy8HF5d9SqXtrmUa2PL6LekCp/e77v0845PIKZDlcX3S+PA06umceBLX6xnZdZeXrutB7HNqq4+YoInqnYkraPr0Tq6/EcxJSXK/vzCYxJLWUcyR4qKA2+wklxLFiISCbwO9AdygGQRSVLV9KNjVPWPfuMfBPyvizysqt3dis+EJlVl7OKxCMJTFz5VdsFw4cuQkQQDnoWOfasyRFLmOI0DB17n+s/6Kn07by/I5I4LO3B911NsWWLCWkSE0Ng5FRWL918S3Lw+rBewUVUzVbUAmA7ceJLxQ4FpJ3nf1ABJm5JYvG0xI3uOpHXD1qUP2vAV/Gus7wayi0ZUbYBA7s4t1IpoQrve7n6Xyfn5EI98tJoubU7jyetqVrHahB43k0UbINvvdY6z7AQi0gGIBb72WxwlIstFZImI3FTGevc4Y5bv3LkzWHEbj+w6vIsXk1+kR4seDI4fXPqgPZkwc7jvvoEbXq3yewVyU9dTWLKL5jEBWqNXUkFRCSOmrqKkRHn9NqtTGO+5mSxK+xSXdQfgEGCGqn+7Tto7N4rcBrwiImeesDHVt1U1QVUTmjcvx7SaJqQ9v/R5DhcdZszFY4iQUn41Cw7C9NsBcQraVX9onjxtFgBd+7l7L8ff5q0jJXsvLw7sSoem3p+CMMbNZJED+H/9akvZU7EO4bhTUKq61fk3E/iWY+sZppr5Outr5m+Zz33d7qNjdMcTB6jCrBGwMwMGvuebAtUD2VmbEGn4a+NAF3yRlsu7C3/kzovP4JrzyjgVZ0wVczNZJANxIhIrInXwJYSk4weJSDwQAyz2WxYjInWd582APkD68eua6iGvII/nljxHp5hO3NXlrtIHLXoV0j6Gfk/DWf2qNkDH0caBMQ1OL1/jwArI3nOIRz9aTde20Yy+9mxXfoYxFeHa1VCqWiQiI4Av8F06+56qponIWGC5qh5NHEOB6Xpsk6rOwFsiUoIvob3gfxWVqV7GrRjHrvxdjL9iPLUjSukeuukb+CoRzrkR+oys+gAdyybOBIqITwhC48BS+OoUKwF4/bbzw+bOXlMzuHqfharOBeYet+zp416PKWW9Rfgm2jLVXHJuMjN+mMGwc4ZxbrNS7lv4eTPMuAuaxcONb3ja/G5TWipQh5533OzK9v86N4PVOfv4x+09adek6m4wNKY8ak5rRRNy8ovyGbNoDG0btuWBHg+cOKDgEHxwO2gJDJniaTO8wiNHyDuyjYZ1W7vSOPDz1G1MXLSZ4X1iubpLq6Bv35jKsnYfxjNvrH6DrLwsJgyYQL1ax93ZqgqfPQS5a+G2D30T8XgoZUqS0zgw+HWErN2H+NOMNXRr15jHr7E6hQlNdmRhPJG+O53JaZO5Je4WerfufeKAJW9C6odwxZPQaUDVB3icjCXLgAh63zkwqNs9UlTMA1NXIgKvDe1Ro+ZRMOHFjixMlSssKSRxUSIxUTE83PPhEwf8uADm/xnOvh4ueaTqAyzF7v0/EVWrBdHtgnsp63NzMkj9aR/v/EeC1SlMSLNkYarcpLRJrNuzjpf7vkx03ehj39ybDR/d6ZsS9OZ/QIT337R/mLfA1ziwTXAbB85Zs43Ji7dw9yWx9D+nZVC3bUywef9JNDXK5n2beTPlTfp36M+VHY6f3e2wr6BdXAhDpobMnNCr5h5tHPjboG1z866DjJq5hh7tGzPK6hQmDNiRhakyJVpC4qJE6taqyxO9nzj2TVWY/UfYlgJDP4BmZ3kTZCm278yiVkQT2vbqGpTt5RcWc/+UldSKFF677XxqR9p3NhP67LfUVJkZP8xg5Y6VPJbwGM3qNTv2zWVvw+pp0PcJiL/amwBL4UbjwL/MTid9237+fmu3U5qlzRgvWbIwVSL3YC7jVoyjd+ve3HTWcU2ENy+EeaMh/lq47DFvAizDL40D+/8mKNtLWr2VKUuzuPeyjvTrbHUKEz4sWRjXqSrPLXmO4pJiEi9KPHZCo3058OEwaNIxZAra/rKzNvoaB95Q+X5UmTsPMHrmGnp2iOHRq+KDEJ0xVSe0PpmmWvpi8xd8m/MtI3qMoF0jv9M5hfnwwR1QdMRX0I6KLnsjHjiQuzNojQOP1inq1Irg1aE9rE5hwo4VuI2r9ubv5fllz9OlaRd+3/n3v76hCnMega0rYfAUaN7JuyDLsGzSTKCYsy/oWeltPfNZOuty8/jnXRdwutUpTBiyZGFc9WLyi+w/sp+3+79NrQi/X7fl70LK/8Flf4LO13sX4ElsSk8D6nD+7aVO1Fhus1J+YtqyLP6775lcHt8iOMEZU8XsWNi4ZuFPC/ks8zOGnzec+CZ+5+i3LIbPR0HcAOg72rsATyJYjQM37jjA6I9T6XVGEx7pH3pHT8aUlyUL44pDhYcYu3gssdGx3Nv13l/f2L8VPvwPaNwebnkn5AraR636v1m+xoGdOld4G4cLinlgykqiakcyfmgPalmdwoQxOw1lXDF+1XhyD+Yy6ZpJ1Ims41tYdMSXKAoOwrAkqNfY2yBPImNJMhBB72G/q/A2xiSl8cOOPCbe1YtW0VHBC84YD1iyMEGXsiOFqRlTGRw/mB4t/KZO//xPkJMMgyZDi4p/Y3dbcXExe/Iq1zjw45U5fLA8mxGXn8VvOjUPcoTGVD07LjZBVVBcwJhFY2jZoCUje/pNgbr8n7BiIlzysG961BC2cf5Cp3Fgxwqtv2F7Hk9+spbesU0YeWVckKMzxhuuJgsRuVpE1ovIRhF5vJT3XxaRFOfxg4js9XtvmIhscB7D3IzTBM87qe+wad8mnr7waRrUdgrD2ctg7mNwZj+44s/eBlgOKXP/BUDCwFO/SutQQRH3T1lJ/TpWpzDVi2unoUQkEngd6A/kAMkikqSq6UfHqOof/cY/CPRwnjcBEoEEQIEVzro/uxWvqbwNP29gQuoErut4HZe2vdS3MC/Xd+NddBv43QSIqNzNbVVh+66KNw58elYaG3ce4P3hvWl5mtUpTPXh5teeXsBGVc1U1QJgOnCy8w9DgWnO86uAL1V1j5MgvgRCp7ucOUFxSTFjFo2hUe1GjLpglG9hUYGvlceR/b4b7+o38TbIcshdnUFhyS5aNDn1xoEfLc9mxoocHrwijkvimgVewZgw4mayaANk+73OcZadQEQ6ALHA16e6rgkNU9dNZc2uNYzqNYqYqBjfwi9GQ/YSuPE1aNXF2wDLadn0zwDo2r/vKa33w/Y8npq1los6NuWhflanMNWPm8lCSlmmZYwdAsxQ1eJTWVdE7hGR5SKyfOfOnRUM01RWTl4Or656lcvaXsa1sdf6Fq58H5InwMV/gC4Vv/y0quVkbyRCGnL2b68o9zoHj/jqFA3r1uZ/h3YnMqK0X19jwpubySIH8D+WbwtsLWPsEH49BVXudVX1bVVNUNWE5s3t8kQvqCrPLH6GCIngqQuf8nWUzVkBcx6Gjn2hX6LXIZbbL40DG5a/caCq8tSna8nceYDxQ7rTopHVKUz15GaySAbiRCRWROrgSwhJxw8SkXggBljst/gLYICIxIhIDDDAWWZCzKxNs1iybQkjzx9Jqwat4MAO39SojVrBwH9CZPjcyrN04gygmPiEhHKv89HyHD5e9RMP9evExWdZncJUX659klW1SERG4PsjHwm8p6ppIjIWWK6qRxPHUGC6qqrfuntE5C/4Eg7AWFXd41aspmJ2Hd7FS8kvcX6L8xkUP8g3d/ZHd8Lhn+E/54dFQdtfZsbRxoHluw9kXe5+npq1lkvOasaIK0JnGlhj3ODq1z5VnQvMPW7Z08e9HlPGuu8B77kWnKm055c+z+GiwyRenEiERMD80bDl33DLBGgdnPmqq0rhoXzyjuSWu3HgAadOcVq92rw82OoUpvqzO4ZMhXyd9TXzt8znvm730TG6I6RMg6X/gAsfgK63eh3eKVs1LQnVfDrGB25Doqo8+Ukqm3cdZPyQHjRvVLcKIjTGW5YszCnbX7Cf55Y8R6eYTtzV5S7Yugpmj4QzLoX+Y70Or0IyliwDIug1LHCim56czayUrfzxyk5cdGZT94MzJgSET/XRhIxxy8exK38X468YT+3D+3x3aNdvBrdODKuC9lG/Ng5sSXTblicdm751P4lJaVwa14wHLrc6hak5wu+TbTyVnJvMzA0zufPcOzk3Jh7evwkO7oTh86BBeF4NtHHe95RoHm3bnrzOkpdfyANTVxJTvzavDO5OhNUpTA1iycKUW35RPmMWjaFdo3bc3/1++CoRNn8PN78Fp/cIvIEQtWqer3FAwqCyGweqKqM/TmXL7oNMv+cimja0OoWpWSxZmHJ7Y/UbZOVlMWHABOqlz4bFr0Hv+6DbEK9Dq5Qdu7ZQK6IJbXqeV+aYKUuzmL1mG49dFU+v2PC6JNiYYLACtymX9N3pTE6bzC1xt9CbepD0IHToAwOe9Tq0SvE1DtxNi6ZlNw5c+9M+xs5Op298c/77N2dWYXTGhA47sjABFZYUkrgokSZRTXik810w8be+G+5unQiRtb0Or1KWTfPdG9qt/+Wlvr/fqVM0qV+HcYOsTmFqLksWJqBJaZNYt2cdr1z2d05L+gMc2A7DP4eGLbwOrdJysjcRIQ2Jv/7EZKGqjJ6ZSs7Ph/ngngtp0qCOBxEaExrsNJQ5qR/3/cibKW/Sv0N/+m34Hn78Dq4fB216eh1apeVt3cHhou3ENGxTauPA95dsYU6qr06RcIbVKUzNZsnClKlESxizaAx1a9XlieiusGg8XHA39Ljd69CCYtmkmUAxZ19wYuPA1Jx9PDs7gyvObsE9l1ZsLm5jqhNLFqZMM36YwcodK3ksbijN5o6GdhfCVc97HVbQbFqXBtSl5x03H7N83+FC7p+6gmYN6/D3W7tZncIYrGZhypB7MJdxK8bRu0VPbvr3OxAVDYMmQ63qcd6+8FA+B47k0qhuK2rX/3UOClVl1Iw1bNubzwf3XkSM1SmMASxZmFKoKs8ueZbikmISd+5G9m2Fuz6HRidvhRFOVk2ZhWo+sfHnHLN84qLNzEvL5clrO9OzQ4xH0RkTeuw0lDnBvM3z+C7nO0Y06ES7zAVw3f9Auwu8DiuoMpYm42scOPCXZauz9/LXuRlc2bkFd18a611wxoQgSxbmGHvz9/LCshfoUr8Nt6+eDT3v9D2qkeLiYvYcOLZx4L5DvvspWjSK4n9u7eabHtYY8ws7DWWO8WLyi+w/so+3czOJbHsBXPOi1yEF3YbPv3MaB3YDfKfdHp2xmu378/nw3otoXN/qFMYcz44szC8W/rSQzzI/Y/jhEuIj6sGg96FW9WuYlzLvGwASBv0WgHcX/siX6dt5/JrO9GhvdQpjSmNHFgaAg4UHGbt4LLHU4d7tm+HOOXBaa6/DcsWO3VnUimhKm57nsirrZ174fB0DzmnJ8D5neB2aMSHL1SMLEblaRNaLyEYRebyMMYNEJF1E0kRkqt/yYhFJcR5JbsZpYPzK8eQe3MbYrVnUueZv0P5Cr0NyxbZV6RSW7KZl03bsPVTAiKmraBUdxUsDrU5hzMm4dmQhIpHA60B/IAdIFpEkVU33GxMHjAb6qOrPIuLfbOiwqnZ3Kz7zq5QdKUxbN40h+/Lofs4gSPhPr0NyTfIHvu8dXQdczqMfrWZHXj4z7ruY6Prh3RDRGLe5eWTRC9ioqpmqWgBMB248bsx/Aa+r6s8AqrrDxXhMKQqKC0j8fjQti4p5KKoDXPt3qMbfsLOzNxEhjfi+UXu+ytjBk9d2plu7xl6HZUzIczNZtAGy/V7nOMv8dQI6ici/RWSJiFzt916UiCx3lt/kYpw12jurXiPzQA5PHyikweApUDsq8EphKm/rDvKLdtCoXmte+nID13RpxbCLz/A6LGPCgpsF7tK+nmopPz8O6Au0Bb4XkS6quhdor6pbRaQj8LWIpKrqpmN+gMg9wD0A7du3D3b81d6G3euZkDaR6w4c4tIbJ0N0W69DctXSiTOAYtbXacXpjevxt4FdrU5hTDm5eWSRA/hPP9YW2FrKmFmqWqiqPwLr8SUPVHWr828m8C1wwiTPqvq2qiaoakLz5s2DvwfVWHFJMYnz76VRcRGjuj8IZ/TxOiTXZa5PB+oyp/HZvH7b+ZwWZXUKY8rLzWSRDMSJSKyI1AGGAMdf1fQpcDmAiDTDd1oqU0RiRKSu3/I+QDomaKYseIrUgt083rAzMRc/5HU4ris8lE9e/jYipDmP39SV89pGex2SMWHFtdNQqlokIiOAL4BI4D1VTRORscByVU1y3hsgIulAMfCYqu4WkYuBt0SkBF9Ce8H/KqpgSl+ygC9e+Ycbmw55wzmP9QjrP64e81OcnAJHoGkH7riwg9fBGBN2XL0pT1XnAnOPW/a033MFHnYe/mMWAee5GdtRtaPqEynV7y7l8qlFTbqJv3atJgx64m6rUxhTATX+Du647gnETfun12EYY0xIqzlfK40xxlSYJQtjjDEBWbIwxhgTkCULY4wxAVmyMMYYE5AlC2OMMQFZsjDGGBOQJQtjjDEBie8m6vAnIjuBLZXYRDNgV5DC8VJ12Q+wfQlV1WVfqst+QOX2pYOqBuzEWm2SRWWJyHJVTfA6jsqqLvsBti+hqrrsS3XZD6iafbHTUMYYYwKyZGGMMSYgSxa/etvrAIKkuuwH2L6EquqyL9VlP6AK9sVqFsYYYwKyIwtjjDEBWbJwiMhfRGSNiKSIyHwROd3rmCpKRF4SkXXO/nwiIo29jqmiRORWEUkTkRIRCbsrV0TkahFZLyIbReRxr+OpDBF5T0R2iMhar2OpDBFpJyLfiEiG87sVtvMKi0iUiCwTkdXOvjzj2s+y01A+InKaqu53nv8BOEdV7/M4rAoRkQHA187Utn8DUNVRHodVISLSGSgB3gIeVdXlHodUbiISCfwA9Ady8M1LP9StKYLdJiKXAQeAyaraxet4KkpEWgOtVXWliDQCVgA3heP/i/imfWygqgdEpDawEHhIVZcE+2fZkYXjaKJwNMA3aXNYUtX5qlrkvFwCtPUynspQ1QxVXe91HBXUC9ioqpmqWgBMB270OKYKU9UFwB6v46gsVd2mqiud53lABtDG26gqRn0OOC9rOw9X/nZZsvAjIs+JSDbwe+DpQOPDxHDgc6+DqKHaANl+r3MI0z9K1ZWInAH0AJZ6G0nFiUikiKQAO4AvVdWVfalRyUJEvhKRtaU8bgRQ1SdVtR0wBRjhbbQnF2hfnDFPAkX49idklWdfwpSUsixsj1irGxFpCMwERh53ZiGsqGqxqnbHdwahl4i4coqwlhsbDVWqemU5h04F5gCJLoZTKYH2RUSGAdcD/TTEC1On8P8SbnKAdn6v2wJbPYrF+HHO788Epqjqx17HEwyquldEvgWuBoJ+EUKNOrI4GRGJ83t5A7DOq1gqS0SuBkYBN6jqIa/jqcGSgTgRiRWROsAQIMnjmGo8pyj8LpChquO8jqcyRKT50asdRaQecCUu/e2yq6EcIjITiMd35c0W4D5V/cnbqCpGRDYCdYHdzqIlYXxl183Aq0BzYC+QoqpXeRtV+YnItcArQCTwnqo+53FIFSYi04C++DqcbgcSVfVdT4OqABG5BPgeSMX3eQd4QlXnehdVxYhIV2ASvt+vCOBDVR3rys+yZGGMMSYQOw1ljDEmIEsWxhhjArJkYYwxJiBLFsYYYwKyZGGMMSYgSxbGnAIRORB41EnXnyEiHZ3nDUXkLRHZ5HQMXSAivUWkjvO8Rt00a0KbJQtjqoiInAtEqmqms2gCvsZ8cap6LnAn0MxpOvgvYLAngRpTCksWxlSA+Lzk9LBKFZHBzvIIEXnDOVKYLSJzRWSgs9rvgVnOuDOB3sCfVbUEwOlOO8cZ+6kz3piQYIe5xlTMLUB3oBu+O5qTRWQB0Ac4AzgPaIGv/fV7zjp9gGnO83Px3Y1eXMb21wIXuBK5MRVgRxbGVMwlwDSn4+d24Dt8f9wvAT5S1RJVzQW+8VunNbCzPBt3kkiBMzmPMZ6zZGFMxZTWfvxkywEOA1HO8zSgm4ic7DNYF8ivQGzGBJ0lC2MqZgEw2Jl4pjlwGbAM37SWv3NqFy3xNd47KgM4C0BVNwHLgWecLqiISNzROTxEpCmwU1ULq2qHjDkZSxbGVMwnwBpgNfA18CfntNNMfPNYrMU3b/hSYJ+zzhyOTR53A62AjSKSCrzDr/NdXA6EXRdUU31Z11ljgkxEGqrqAefoYBnQR1VznfkGvnFel1XYPrqNj4HRYTz/uKlm7GooY4JvtjMhTR3gL84RB6p6WEQS8c3DnVXWys5ESZ9aojChxI4sjDHGBGQ1C2OMMQFZsjDGGBOQJQtjjDEBWbIwxhgTkCULY4wxAVmyMMYYE9D/Ayy0PawJBkxmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x23ebd8dddd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "accuracy_s1 =np.array(accuracy_s).reshape(len(C_s),len(gamma_s))\n",
    "x_axis = np.log10(C_s)\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    pyplot.plot(x_axis, np.array(accuracy_s1[:,j]), label = ' Test - log(gamma)' + str(np.log10(gamma)))\n",
    "\n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'log(C)' )                                                                                                      \n",
    "pyplot.ylabel( 'accuracy' )\n",
    "pyplot.savefig('RBF_SVM_Otto.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "最后可以很直观的看出已经达到了1.0的效果，说明参数设定较为合理\n",
    "该处出现了报错的问题，经过查询发现参数设置的区间应为偶数才可以正常运行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作业总结"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "内容小结：\n",
    "1）对上周学习内容进行了复习，根据数据特点，对数据进行了适当的特征工程和数据分析；\n",
    "2）对上周没有用到的随机数据分割进行了练习，使用两种参数进行了两次数据分割；\n",
    "3）熟悉了Logistic回归的使用方法，对GridSearchCV和LogisticRegressionCV都进行了练习，并且有一个简单的对比；\n",
    "4）熟悉了线性SVM的使用方法，并且与Logistic回归做了简单的对比；\n",
    "5）对RBF核的SVM进行了练习，并且根据数据对参数进行了对应的调整；\n",
    "6) 解决问题的能力有了一定的提高，代码中出现的问题基本可以通过666法则自行解决。\n",
    "\n",
    "问题小结：\n",
    "1）由于知识所限，对数据集中的数据的关联性不太了解；\n",
    "2）虽然了解混淆矩阵的大致意思，但是还是不太会对其进行分析；\n",
    "3）只是简单对方法进行了重现，并没有更加深入研究相关内容；\n",
    "4）对于理论问题还要不了解的地方，还需要反复学习，深入了解。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
